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Introduction 



Science is not a subject that you may think about often, but it is 
part of our lives in many ways. Results of research studies are re- 
ported in the news. As a student or as part of your job, you may need 
to read research papers or research reports, and you need to under- 
stand what was done and why. You also need to be able to decide 
whether the conclusions of a study were supported by the results of 
the research. You may also do research yourself, either as an assign- 
ment for a class or as part of your job. You might need to be able to 
plan and carry out a research project. There are therefore a number of 
reasons that you might need to understand how research is done as 
well as how to evaluate the results of research. 

Unfortunately, not all of the research that is published or reported 
is good research. There can be various problems with the ways that 
research is carried out. If you read research without being critical of 
it, you can be misled if the research is not done well. 

Content and Purpose 

In this book, we have discussed the ways in which we know what 
we believe we know, what science is, and what social science is. So- 
cial scientists use research in an effort to look objectively at the ways 
in which people behave, think, and interact. With this background, 
we discuss two major approaches to research: quantitative research, 
which involves converting observations to numbers and analyzing 
them statistically, and qualitative research, which looks at participants’ 
opinions, behaviors and experiences from their own points of view 
and in a more subjective way. We have put particular emphasis on 
research related to communication and language teaching/leaming. 
These are important areas of research, because the study of communi- 
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cation can potentially help people communicate more effectively and 
can ultimately improve human relationships, while the study of lan- 
guage teaching/leaming can lead to improve methods for teachers and 
more effective use of strategies by learners. 

Therefore, the purpose of this book is to help readers understand 
the characteristics of science in general and social science in particu- 
lar and how these influence the ways research is done. This book is 
also intended to help readers understand and evaluate research they 
read and to know how to plan a research study, if they want to do so. 
We have tried to give a comprehensive view of social science research, 
beginning with its foundations in social science, continuing through 
practical discussions of quantitative and qualitative approaches, and 
concluding with advice on how to report on the results of research 
through papers and presentations. 

Sections of the Book 

In the first section, “Looking at Science,” there are chapters on 
what science is, its purposes, how we know what we know, and how 
science is different from other ways of knowing. In this section, we 
bring up some of the issues and problems that researchers face, which 
are explained in more detail in later chapters. We also look at what 
research is and how qualitative and quantitative approaches to research 
are different. 

In the next section, “Quantitative Research,” we look at the basic 
concepts of quantitative research, including variables and hypotheses; 
and methods for collecting data, including questionnaires, interviews, 
and observation, which can be applied to both quantitative and quali- 
tative research. We also look at validity and reliability in quantitative 
research and how the results of quantitative research are analyzed. 

In the third section, “Qualitative Research,” we discuss aspects of 
qualitative research, including case studies, how data is gathered in 
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qualitative research, and how to analyze the data gathered. 

The sections on quantitative and qualitative research are intended 
to be useful both for readers who want to carry out research and those 
who want to understand how it is done, for example, in order to evaluate 
research that they read about. 

If you do research, it is also important to report effectively on the 
results of your research. In the appendices, there is information on 
writing research papers, including how to avoid plagiarism; a sample 
research paper; and advice on doing presentations of research and on 
writing titles, abstracts, and biodata for a research paper or presenta- 
tion. 



Organization of the Chapters 

Each chapter begins with a list of keywords that are used in the 
reading, introducing the main concepts of that chapter. The reading is 
followed by application questions. These questions give readers an 
opportunity to apply the concepts from the chapter to an area of social 
science that they are interested in. These questions can also be used to 
guide readers through the steps of doing a research project of their 
own, as they form hypotheses or research questions, write question- 
naires and interview protocols, and analyze the results. 

Many chapters also have discussion questions, in which readers 
consider the sample research paper found in Appendix III in terms of 
the concepts related to that chapter. These are intended to help readers 
see how the concepts were applied in a specific study. It may be use- 
ful to read the paper in Appendix III before reading Chapter 4 in order 
to be familiar with it, and then to refer back to it when answering the 
discussion questions through the rest of the book. Suggested answers 
for these questions are found in Appendix VI. 
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Other Resources 

While this book deals to some extent with language teaching, it 
includes little information about language testing, which is often an 
important aspect of doing research related to language teaching and 
learning. If you are interested in language testing, we have written 
another book on that subject. It is: 

Kitao, S. K. & Kitao, K. (1999). Essentials of English language test- 
ing. Tokyo: Eichosha. 

Supplementary material, including a bibliography on social science 
research, and useful links related to this book can be found at: 

http://ilc2.doshisha.ac.jp/users/kkitao/library/resource/research/ 
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What is Science? 
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In this chapter, we will discuss what science and the scientific 
method are. Science can be difficult to define, but we will discuss 
some of the purposes and characteristics of it. We will also compare 
natural science, with which you may be more familiar, with social 
science. 



Natural and Social Sciences 

Characteristics of Natural Sciences 

You may be familiar with natural sciences like biology, medicine, 
chemistry, and astronomy. Natural science has made great contribu- 
tions to our lives. It has improved our lives in many ways (though of 
course it has also caused problems). Natural science has found ways 
to cure diseases. Over the centuries, science has continued to develop, 
and it has corrected and built on previous knowledge. For example, 
centuries ago, people did not know what caused diseases or how they 
were spread. They had a variety of beliefs about where diseases came 
from. In some cultures, diseases were believed to be a punishment 
from God; in other cultures, they were believed to be caused by a 
curse from an enemy. 

Eventually scientists learned that viruses and bacteria cause dis- 
eases. They learned how to keep disease from spreading, how to diag- 
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nose disease, how to cure many diseases, and how to prevent many 
diseases through vaccination. However, for some diseases there is no 
cure yet, or no way to prevent them. In addition, new diseases de- 
velop, and old ones change, so scientists still study diseases and how 
to prevent or cure them. 

Natural scientists observe the natural world, whether it is the move- 
ment of the stars or the progress of diseases in the human body, and 
study and describe what they see. They find laws to describe the phe- 
nomena they are studying. An astronomer might study the movement 
of a planet and predict how it will move in the future. Following sci- 
entific laws, he/she can do so precisely. 

Social Sciences and Their Relation to Natural Sciences 

Social sciences are fields of study that deal with human behavior, 
thinking, and relationships. The social sciences are similar to natural 
sciences in some ways but different in others. In general, both social 
and natural scientists test theories. That is, they do research to find 
out whether certain theories are supported or not. They use these theo- 
ries to explain why things happen and to try to predict what will hap- 
pen in the future. That is the ultimate goal of science — to be able to 
explain and predict. 

Laws vs. generalizations. Social scientists study human behavior 
and describe and try to understand what they see, just as natural sci- 
entists study natural phenomenon and also describe and try to under- 
stand what they see. However, it is difficult to develop laws to de- 
scribe human behavior; human behavior is too unpredictable. A chem- 
ist, for example, knows that if you mix two chemicals, you will al- 
ways get the same result. You do not get different results under the 
same conditions at different times, and a chemist can explain why 
such a reaction occurs. Social science, in contrast, is not so exact. It is 
much more difficult to explain human behavior and make predictions 
about it. Social scientists do try to understand human behavior, make 
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some generalizations about it, and use these generalizations to make 
predictions. However, social sciences are not as clear-cut or precise 
as natural sciences. 

Measurement. Another difference between natural and social sci- 
ence is that there is often no precise way to measure the kinds of 
phenomena that social scientists study. Given the necessary instru- 
ments, a natural scientist can accurately measure or test the move- 
ment of the planets, how strong a muscle is, or the type of bacteria 
that is causing a disease. This is true for many of the phenomena that 
natural scientists study. 

However, the types of phenomena that social scientists study are 
often more difficult to measure, because they are often inside the minds 
of the participants in the study. They cannot be seen or measured in 
any direct way, or with a scientific instrument. A social scientist might 
be studying the effectiveness of advertising for a new drink and want 
to measure the participants’ reactions to different advertisements. That 
reaction is not something that can be measured with a scale, as weight 
is measured. The social scientist might ask the participants questions 
about their attitude is toward the drink or whether they plan to buy the 
drink. 

One of the problems with this type of measure is that the scientist 
doing the research has to depend on what the participants say they 
think or how they say they will act. This is an indirect way to measure 
the effect. It may be that some of the participants will try to tell the 
researcher what he/she wants to hear rather than what they really think. 

Also, the researcher might try to find out whether the participants 
really did buy the drink after seeing the advertisement. That would be 
a direct way to measure the effect, but there might also be practical 
problems. Is there any way for the scientist to find out whether the 
participants bought the new drink, other than just to ask them? 

In designing research, the researcher must plan the best way to 
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gather the information. This, however, might involve some compro- 
mise, because the best way might be too expensive, too time-consum- 
ing, or impractical in other ways. For example, the researcher might 
be thinking about whether to gather the information through a ques- 
tionnaire or an interview. The researcher might think that an inter- 
view is better, because in an interview, it is easier to get more infor- 
mation if the participant does not give enough information. However, 
interviews take much more time than questionnaires, which can be 
used to gather a lot of data in a short time. Therefore, the interviewer 
might compromise and use questionnaires instead. 

There might be other practical problems. As mentioned above, 
sometimes participants only say what the researcher wants to hear, or 
they might answer in a way they think makes them look better. For 
example, if you ask people whether they wash their hands after using 
the toilet, they might say that they do, even though they do not, be- 
cause they might be embarrassed to tell the truth. 

In order to find out what people really do, a researcher might want 
to observe people in a public restroom. However, it is unlikely that a 
researcher could get permission to put a hidden camera in a public 
restroom. The researcher might stand in the restroom and watch people 
to see whether they wash their hands. The problem with this is that 
people might behave differently if they know they are being observed. 
Another problem is whether the behavior of people in a public restroom 
can be generalized to their behavior at home. If the researcher does 
figure out a way to observe people in a public restroom, does their 
behavior only apply to the public restroom, or do they behave the 
same way at home? 

Therefore, one of the most important issues in doing scientific re- 
search is in evaluating the strengths and weaknesses of different ways 
of gathering information and deciding which one is most appropriate 
for the practical, financial and time constraints of a study. Among the 
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questions a researcher must ask is whether people will tell the truth 
about what they do or think, and what kinds of questions can best get 
the information that the researcher wants. The researcher also needs 
to consider what kind of observation might help get the information 
and if observation is possible, how to do it. In addition, the researcher 
needs to recognize any weaknesses in the chosen method and con- 
sider the effects they might have. 

Conclusion 

The purpose of science is to explain phenomena and to try to pre- 
dict them. Natural science has a long history. Social science devel- 
oped using natural science as a model, because at the time it was the 
only model for science. Though they are similar, there are differences 
as well, for example, in that social science typically deals with gener- 
alizations rather than laws, and in the difficulty of measuring phe- 
nomena in social science. 

Application Questions 

Think of a field of natural science. What does it try to understand, 
explain, or predict? In what ways, if any, has it influenced the lives of 
people? For example, among the things that biology tries to under- 
stand is how plants grow, and how they can be grown better by pro- 
viding fertilizer, killing insects that eat them, etc. This has benefited people 
in that it allows more food to be produced from a smaller land area. 

As mentioned in this chapter, measurement in the fields of social 
science can be difficult, because many of the concepts that social sci- 
entists study are inside the mind. Think about the following concepts. 
How do you think you could measure them? 

1 . the closeness of a friendship 

2. the effectiveness of an advertisement 

3. what viewers like about a television show 
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For example, you might measure the closeness of a friendship by 
making a questionnaire based on the characteristics of friendships. 
Since a characteristic of friendship is trust, you might ask if the par- 
ticipant would tell the friend a secret or lend the friend a large sum of 
money. 



Chapter 2 

Aims and Methods of Science 
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The purpose of science is to explain various phenomena. In order 
to do that, scientists use what has come to be called the scientific 
method. The scientific method is a series of steps that scientists use to 
test their ideas. In this chapter, we will discuss the purpose of what 
scientists do and how they go about doing it. 



Whether natural or social, the ultimate aim of science is to develop 
theories to try to understand how phenomena in the world work and 
to make predictions based on this understanding. A theory is a series 
of interrelated ideas that specify the relationships among the ideas 
and which can be used to explain and predict. For example, an inter- 
cultural communication scientist might be interested in friendships 
between Americans and Japanese and how cultural differences influ- 
ence the development of friendships. A scientist interested in political 
communication might wonder why one speech by a politician causes 
voters to vote for him/her, and another does not. A scientist interested 
in persuasion might want to know what kinds of persuasive messages 
are most effective in different situations or with different groups of 
people and why. 

Scientists develop theories in order to try to explain the relation- 
ships among two or more factors and to make predictions about what 
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will happen in different situations. For example, a social scientist might 
be interested in education and what factors lead to success in school. 
The scientist might be interested in such factors at intelligence, verbal 
ability, anxiety, motivation, family background, and social class. He/ 
she would look at that the relationships among those six factors and 
how they contribute to success or failure in school. These explana- 
tions would be used to try to predict, to some extent, success or fail- 
ure in school. Sometimes scientists may look at only a very small part 
of a theory. For example, one scientist may look just at the relation- 
ships between motivation and success or failure in school. 

One important characteristic of this type of work is that it is gen- 
eral. Scientists who are looking at success or failure in school look for 
explanations as to why children in general do well or poorly in school. 
They are not looking just at why one specific child does well or poorly. 
The aim of science is to develop a theory that applies generally to 
many, if not all, children, rather than only to one specific child. 

The Scientific Method 

One of the basics of science is the use of the scientific method. It is 
a process that scientists go through when they do research. 

Problem/Obstacle/Idea 

A scientist starts by wondering about something. The scientist may 
wonder about a problem, for example, why do some language stu- 
dents learn the language better than others? The scientist may be try- 
ing to explain or understand a phenomenon. Why do some interper- 
sonal friendships grow closer than others? 

Forming a Hypothesis 

Next, a scientist forms a hypothesis about the relationship between 
two or more ideas. For example, a hypothesis might be that students 
with high motivation do better in school than students who have low 
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motivation. A political speech that includes references to voters’ eco- 
nomic problems will convince more voters than a speech that does 
not. Forming a hypothesis may require reading about the research 
that other scientists have done and the theories they have developed. 
This process emphasizes the cumulative nature of science. No scien- 
tist works completely alone. All scientists look at the work that other 
scientists have done and build on it. 

Wondering about a problem or an idea and forming a hypothesis 
are activities that are not limited to science. People do these things 
informally in everyday life as well. A teacher might wonder how to 
help students learn better and form a hypothesis that praising students 
helps them learn better. 

Observation/Test/Experiment 

In social science, the next step is to test the relationships between 
the concepts stated in the hypothesis, that is, to do research to find out 
whether the hypothesis is supported or not. In order to test the hy- 
pothesis, the researcher needs to decide how to measure the concepts. 
For example, the scientist might be testing the hypothesis that stu- 
dents who are more motivated, that is, students who have a greater 
desire to learn, do better in school. 

In that case, the researcher needs to figure out a way to measure 
motivation. Motivation might be measured by giving students a ques- 
tionnaire with questions about how much they want to learn or how 
much time they spend on homework. It might be measured by obser- 
vation, that is, the researcher might watch the children and rate them 
according to how much they participate in the class, assuming that 
the students who participate most are the most motivated. The re- 
searcher might ask the students’ teacher to rate the students according 
to their motivation. 

The researcher also has to decide how to measure the students’ 
achievement, that is, how well they do in school. This might be 
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measured by grades or by an achievement test. Then the researcher 
would look at the relationship between motivation and achievement, 
based on the results of the measurements. 

A difference between scientists doing research and individuals 
considering the world around them in everyday situations is that sci- 
entists test their hypotheses in a systematic, controlled way. That is 
scientific research. This topic will be discussed in more detail in the 
next chapter. 

Evaluation/Feedback/Accumulation 

Finally, the researcher evaluates the results and thinks about what 
they mean and how they might relate to other ideas or lead to new 
ideas. Perhaps the researcher would find that there is a relationship 
between motivation and achievement in school but not a strong one. 
Motivation alone does not explain achievement in school. 

The researcher’s next step might be to look at other factors. The 
researcher would be interested not only in what other factors might 
influence achievement but also in how those other factors are related 
to motivation. What role does intelligence play? Does motivation have 
more effect for less intelligent students than for more intelligent stu- 
dents? And what role does family background play ? Are students whose 
parents place a high value on education more motivated? 

Some researchers are also interested in practical application of their 
research results. If students who are more motivated learn better, what 
can be done to increase the motivation of the less motivated students? 
If teachers praise students, does that increase their motivation? Are 
certain types of teaching materials more motivating than others? 

The researcher publishes the study and other researchers read it. 
They may look for weaknesses in the study and try to design better 
studies. They may agree with the results but wonder whether the same 
results would be obtained from a different group of students. Does 
motivation work in the same way with 18 -year-olds as it does with 
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six-year-olds? Does it work the same way in a language class as it 
does in a literature class? A science class? Another scientist might 
also study these questions. In this way, the results of research studies 
build on one another, and knowledge grows. 

The scientific method is therefore a circular process. Each research 
project will hopefully answer some questions — but the results of the 
research might raise other questions. The researcher, or other research- 
ers involved in researching the same subject, then start the process 
again. What kinds of relationships or questions are suggested by the 
results of research? How might they be tested? 

Conclusion 

The purpose of science is to explain phenomena and try to predict 
what will happen. In order to do this, researchers go to through the 
steps of the scientific method. The researcher begins by wondering 
how something works, or why something happens, and then forms a 
hypothesis or research questions about it. The researcher then carries 
out a study to test the hypothesis. Using the results of the research, the 
researcher, or others who read the results, form new questions and 
knowledge is accumulated. 

Application Questions 

Think of a social science-related question that you would be inter- 
ested in. If you were a social scientist, how do you think you could do 
a study to find an answer to it? What might be the advantages and 
disadvantages of doing the study that way? 

For example, you might be interested in how people choose their 
level of politeness when they make a request. Do speakers choose 
much more polite strategies when they make a large request than when 
they make a small request? In order to study the answers to these 
questions, you might observe people making requests in natural situ- 
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ations and compare how people make large and small requests. The 
biggest advantage of this approach is that you would be able to see 
how people make requests, rather than how they say they make re- 
quests on a questionnaire. However, there are several disadvantages. 
In natural situations, it might be easy to find many small requests, but 
it might be difficult to find large requests. Also, you would only be 
able to observe requests that are made in public situations, but not in 
private situations. In addition, you might make judgments about the 
size of the request (large or small), but you might not be able to find 
out what the speaker her/himself thought about the size of the re- 
quest. 
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In this chapter, we will discuss different types of knowledge, based 
on how we know what we believe we know. We will also discuss how 
scientific knowledge is different from and similar to what most people 
think of as common sense and how scientists approach research. 

Ways of Knowing 

The ways or methods by which we know things can be divided 
into five categories. 

The Method of Tenacity 

The first method is the “method of tenacity,” This means that we 
tend to hold firmly to something we believe to be true, just because 
we have always believed it. Some religious beliefs are like this. Some 
people believe in God this way. There is no way to prove scientifi- 
cally that God does or does not exist, but people choose to believe in 
God, and some people just continue to believe in God because that is 
what they have always believed. 

Superstition often involves this method of knowing. Some people 
believe that going to a shrine on New Year’s Day will bring them 
good luck that year. They might visit a shrine every year, and some 
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years they have good luck, and some years they have bad luck. In 
spite of this, they may continue to “know” that going to a shrine brings 
them luck. 

Experience can often lead to knowing by the method of tenacity. 
For example, you may read one science fiction book and find that you 
do not like it. You may conclude that you do not like science fiction. 
Even if your friends try to convince you to read science fiction books, 
you may refuse, because you do not believe that you like science fic- 
tion, based on your one experience with it. 

The Method of Experience 

Another method of knowing is the “method of experience.” Through 
our experiences, we learn various things about relationships among 
people and how the world works. For example, when you are talking 
with a friend, you might find that if you share some personal informa- 
tion about yourself, the other person also shares personal information 
about him/herself. You might notice this experience once, and then 
you might pay attention to see if it happens again. You might even 
purposely give personal information to a friend to see if she/he re- 
sponds by giving personal information. Based on your experiences, 
you might revise what you know. 

One problem with the method of experience is that it is difficult to 
know whether your experiences are typical. You may have one type 
of experience, but if you talk to another person, you may find that the 
other person has completely different experiences. 

The Method of Authority 

The third method is the “method of authority.” This means that we 
believe something because someone in authority tells us it is true. 
When people express their opinions, they often refer to authorities to 
back them up. 

A great deal of what we know, we know because an expert or some- 
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one we trust has told us that it is true. We might know that the Battle 
of Sekigahara was fought in 1600 because a teacher said so. We might 
know that fog is formed by cold air and warm air coming together 
because we read it in a textbook. We might know that a law will soon 
be changed because a government official told us so. Often the fact 
that information is found in a book or on the Internet or is broadcast 
on TV gives it a great deal of authority. However, the fact that some- 
thing is in print or on TV does not make it true. 

Printed materials seem to carry particular authority. Books, maga- 
zines, and newspapers and now the Internet are thought by some people 
to be especially believable. However, we need to keep in mind that 
just because something is in print, this does not necessarily make it 
true. This is especially true in the case of the Internet, where anyone 
who wants can put up information. It is therefore important to criti- 
cally evaluate information that we read. 

We need to make use of the method of authority in many areas of 
our lives. This is necessary, because we cannot test every fact person- 
ally or check the original sources in all cases. However, as mentioned 
above, not everything we read or hear is tme, and even experts can 
disagree, so knowing on whom to depend can be difficult. We may 
depend on what someone else says, but often we do not know where 
they got the information, or whether their information is reliable. 

The Method of Reason 

The fourth method is the “method of reason.” This method is based 
on reason, that is, on what seems “reasonable.” This might also be 
called “common sense.” It may seem reasonable that if the govern- 
ment provides more money for education, education will improve. 

The problem with this method is that if two people using reason 
might come to different conclusions. For example, suppose Joe and 
Sandra are talking about education. Joe might think it is reasonable that 
the more money is spent on education, the better education will be. 
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Therefore, the only way to improve education is to spend money on it. 

Sandra might argue that education can be improved without spend- 
ing very much money. She might reason that money spent on salaries 
for administrators might not necessarily improve education. In addi- 
tion, she might believe that the most effective methods of improving 
education require little money. Getting parents more involved in their 
children’s education need not be expensive, but it can be an effective way 
to improve education. Joe might respond that spending money on 
administrators’ salaries will improve education, because it will attract 
well-qualified people. In addition, it is hard to say how effective it would 
be to get parents more involved in education. Joe and Sarah cannot 
agree, and by reason alone, there is really no way to know who is right. 

The Method of Science 

The fifth method of knowing is the “method of science.” The 
method of science is a way of looking objectively at knowledge, so 
that anyone who looks at it will see it in the same way. The basic, 
underlying belief of this method of knowing is that we need to test 
what we believe we know. Scientists make studies to test what they 
believe is true. This is research. We have to do research, because rea- 
son alone can bring two people to different conclusions, with no way 
of knowing who is right. 

One advantage of the method of science is that it is self-correcting. 
One researcher does a study and learns something and publishes the 
information in a journal. Other researchers read the paper, and one 
might disagree with the first researcher’s explanation. The second re- 
searcher might do another study and show that the first researcher’s 
explanation was wrong. Therefore, an important characteristic of the 
method of science is that when one researcher makes a mistake, other 
researchers have a chance to find the mistake and correct it. As a reader 
of research papers, you should read critically, that is, you should con- 
sider how the research was done and whether its results support the 
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explanation the researcher gives. 

Science may come to the same conclusion as reason or common 
sense. In fact, hypotheses that scientists use in their research are often 
based, at least in part, on reason, so scientific learning can be built on 
reason. The difference between these two methods of knowing is that 
the conclusions of science are based on research, that is, they are tested. 

Research could be performed to find out whether Joe or Sandra is 
right in their disagreement about spending money on education. For 
example, a study could be done to test whether spending more money 
on administrators’ salaries really attracts better administrators and 
therefore improves education. Another study could test whether in- 
volving parents in their children’s education really improves educa- 
tion without increasing the cost. In these ways, science could be used 
to test the conclusions arrived at by reason. 

The research that scientists do might start out in the library. A re- 
searcher interested in ways to improve education would start out by 
reading about previous studies of education and about theories of edu- 
cation. He/she would use that reading to find out what other research- 
ers have studied and what they have learned, as well as what theorists 
believe about education. However, this “research” (sometimes called 
“secondary research”) is not the kind of research we are discussing 
here. After reading what others have done, the researcher decides what 
he/she wants to study and designs research. (“Designing research” refers 
to planning a way of doing a research project, including deciding how to 
collect information, how to choose participants in the study, etc.) 

Research can be divided into two basic types, experimental and 
non-experimental research. In non-experimental research, the re- 
searcher studies a situation as it is, either at one point in time or over 
time. For example, the researcher studying how involvement of par- 
ents improves education might interview parents about how they are 
involved in their children’s education and then compare the children 
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of highly involved and uninvolved parents to see if they are different. 
If children whose parents were more involved did better in school, the 
researcher might conclude that parental involvement can improve educa- 
tion. In non-experimental research, the researcher does not do anything 
to change the situation; he/she just examines the situation as it is. 

In the case of experimental research, the researcher makes some 
change and then studies the effects of the change. For example, the 
researcher might train some parents at one school on how to most 
effectively help their children with their homework, but other parents 
would not receive any training. The researcher would then look at the 
differences between students whose parents have been trained to help 
with homework and those whose parents have not been trained. If the 
students whose parents had been trained did better in school after 
their parents were trained, but the other students did not, the researcher 
might conclude that training parents to help students with their home- 
work effectively could improve education. 

Differences Between Common Sense and Science 

As mentioned above, science and reason may come to the same 
conclusions. However, they are different in that science involves test- 
ing conclusions in a systematic and controlled way. Every day, we 
make common- sense assumptions about the world. Science has some 
differences with our common-sense view of the world. 

Selecting 

One of the differences between common sense and science is how 
they test what they believe. It might seem to be common sense that 
many people disapprove of the government, because a lot of your 
friends disapprove of the government. However, your friends might 
not be typical of all the people in the country. This evidence is selec- 
tive. In science, however, researchers are careful about how they choose 
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the people they use as evidence. Rather than selecting examples of 
people that they know, researchers would ideally choose people at 
random and ask them what they think of the government. That way, 
the researcher is able to find out what people in general think, rather 
than what a certain group of people thinks. 

Scientists distinguish between what they call a population and a 
sample. A population is all of the people that the researcher is inter- 
ested in, that is, for example, all Americans, all Japanese, all Ameri- 
can university students, etc. The population is all the people to whom 
the researcher wants to generalize the study. Of course, it is impos- 
sible to study all Americans or all Japanese. A sample is the people 
that the researcher actually studies, that is, the ones who fill out the 
questionnaire or the ones he/she observes. One of the problems in 
science is choosing people to participate in the study, that is, to choose 
people for the sample that are not different from the population as a 
whole. For example, if a researcher is interested in how Americans 
behave when they shop, the researcher might study this by observing 
how people make purchases in an expensive store. However, the re- 
sults might be different from what they would have been at a discount 
store. 

Control 

Another difference between research and common sense is that 
researchers try to exercise “control.” Control involves a situation where 
other possible causes can be eliminated. In everyday situations, we 
may not try to eliminate other possible causes. For example, you might 
notice that when you want to talk, your friend always seems to be 
busy and does not have time to talk with you. Perhaps you would 
assume that the person did not like you any more. However, if you 
were looking at it from a scientist’s point of view, you would try to 
eliminate other explanations. Perhaps the friend is particularly busy 
at school this year. Perhaps she is having family problems and just 
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does not feel like talking. 

Avoiding the Metaphysical 

Another difference in the way that scientists look at the world is 
that they do not look at “metaphysical” explanations. Metaphysical 
explanations are ones that cannot be tested. You may believe, for ex- 
ample, that if someone dies, that is fate. However, fate is something 
that cannot be proven or disproven. Therefore, a scientist does not use 
that explanation. The scientist might personally believe in fate, or some 
other metaphysical explanation such as God’s will, but he or she does 
not use those in scientific explanations. Science is about what can be 
observed, not about faith in things that cannot be seen. 

Conclusion 

^1 

We know what we know through various ways, including the 
method of tenacity, the method of experience, the method of author- 
ity, the method of reason, and the method of science. While science 
and reason are related, they are different in that science is based on 
testing conclusions, while there is no way to test the conclusions of 
reason. 

Application Questions 

Think of five things that you know. Which method of knowing 
does each of them fit into? 

For example, you know that if you do not water plants, they will 
die. You may have learned this from your own experience, so you 
know that by the method of experience. You know that in English, the 
plural is formed by adding -s to a word. You may have learned this 
from a textbook, so you know it by the method of authority. 
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Research can be divided into two major approaches — quantita- 
tive research and qualitative research. In this book, we will first look 
at quantitative research and then at qualitative research. In this chap- 
ter, we will look at the differences. 

In quantitative research, the researcher studies a subject by mak- 
ing observations as. objectively as possible and converting those ob- 
servations to numbers and then using statistics to analyze the num- 
bers. Qualitative research is more subjective, and it is more closely 
related to the subjects’ own understanding and experiences. This dif- 
ference may seem difficult to understand, but an example might help. 

Two Sample Studies 

Suppose two researchers. Dr. Stiff and Dr. Miller, are doing stud- 
ies on close friends and the personal information that they discuss. 
They are both looking at the research question “How is the discussion 
of personal information in close friendships different for men and 
women?” Dr. Stiff will use a quantitative approach, and Dr. Miller 
will use a qualitative approach. Even though they are looking at the 
same subject, their approaches are different. In this example, they 
both decide to use questionnaires. Below are parts of their question- 
naires, with the answers of some participants in the studies. 

Dr. Stiff uses a Lickert scale. Part of his questionnaire would look 
like this: 
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Think of a close friend and answer the following questions about 
him/her. Circle 5 if you strongly agree, 4 if you agree, three if you are 
neutral, 2 if you disagree, and 1 if you strongly disagree. 



1 . There is nothing that I cannot tell my friend. 12 3 (5)5 



3. If I have a problem, I tell my friend about it. 1 



Dr. Stiff would take the average response for each of the ques- 
tions, as well as the average score for all questions, and compare them 
for males and females. He would then draw conclusions about how 
men and women are different in the personal information that they 
share with close friends. 

When he wrote about this part of the questionnaire, he might write, 
“The average score for self-disclosure for female participants was 4.3 
and for male subjects was 2.5, indicating that women are more likely 
to discuss personal information with a close friend than men.” 

Dr. Miller uses open-ended questions, that is, questions that do not 
have a “right” or “wrong” answer, which the participant can answer 
any way he/she wants. An example of an open-ended question would 
be, “How do you think friendships between men are different from 
friendships between women?” (Closed-ended questions, in contrast, 
would be questions like “How old are you?” and “Where are you 
from?” which have a definite answer.) Part of Dr. Miller’s question- 
naire would look like this: 



strongly strongly 

disagree agree 



2. My friend tells me all her/his secrets. 



1 




Write about a close friend. How much do you confide in this friend? 
What kind of personal information do you give this friend? Do you 
talk to this friend about personal problems? 
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(answer from a woman) 

My best friend’s name is Sherry. I feel as if there is nothing that I 
can’t tell her. We talk about our plans with the future, our relation- 
ships with our boyfriends, the problems we have with our parents, 
and so on. Whenever I have a problem, she is the first one I want to 
talk to. Sherry doesn’t necessarily offer solutions, but talking to her 
always makes my problems feel smaller, even though nothing has 
really changed. For example, ... 

(answer from a man) 

I have a good friend at work named Bob. Sometimes after work we 
go out for a drink, and we talk about sports mainly. We don’t talk too 
much about personal problems, though once I asked him for advice 
about a problem with my girlfriend. He gave me good advice, and it 
helped solve the problem.... 

Dr. Miller would read all of the answers and look for patterns, 
particularly patterns of differences between men and women. For ex- 
ample, she might notice that many of the women said that just talking 
about their problems with their friend helped them, while many of the 
men seemed more interested in practical help from their friends. 

When Dr. Miller wrote up her study, she might write, “For women, 
talking about the problem appeared to have a comforting effect, even 
if the friend did not offer practical help. As one subject wrote, ‘Talk- 
ing to Sherry about my problems always makes them feel smaller, 
even though nothing has really changed.’ Men, in contrast, seemed 
more interested in getting practical help, such as advice to help solve 
a problem.” 

Differences Between These Two Approaches 

As you can see from these examples, quantitative studies involve 
converting observations into numbers and comparing those numbers. 
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Even if quantitative researchers use open-ended questions, they ana- 
lyze them by counting different types of responses. (For example, 
how many times did subjects say that just talking about problems 
helped them? How many times did subjects say that they wanted prac- 
tical help with their problem?) Qualitative studies, in contrast, involve 
trying to find patterns in the subjects’ responses, and the results are 
often reported using the subjects’ own words. 

An important difference between quantitative and qualitative ap- 
proaches is that, in quantitative approaches, the researcher sets the 
categories and decides what will be important to study. In qualitative 
research, the subjects have more opportunity to “influence” the direc- 
tion of the research by bringing up aspects of their own experience 
that the researcher might not have considered. Dr. Stiff, the quantita- 
tive researcher, may begin a study knowing that he wants to see whether 
talking about personal problems is more important in relationships 
between men than in relationships between women. He designs the 
study, in part, to learn about this, perhaps by asking questions about 
how much subjects talk about personal problems with friends. Dr. 
Miller, the qualitative researcher, on the other hand, might not be look- 
ing for that aspect of friendships, but she might find that many women 
mention this aspect, while few men do. 

In a sense, quantitative research is like taking one or more “photo- 
graphs” of a situation at one or more points in time. A quantitative 
researcher who is studying how friendships change over time may 
have participants who are starting a friendship fill out a question- 
naire, and then have them fill out the questionnaire again after six 
months. Therefore, the researcher sees what the relationships are like 
at the beginning and again what they are like after six months. In this 
way, the researcher can see how the relationships are different at those 
two points in time, although it is difficult to see the process that the 
friendship goes through. However, the advantage of quantitative re- 
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search is that it can include many people. Qualitative research, on the 
other hand, is able to look in more depth at the process that a relation- 
ship goes through, though a qualitative study would use fewer people. 
A qualitative researcher might interview people in detail over the first 
six months of their relationship to learn how the relationship is chang- 
ing. In that way, a qualitative study is more like a video tape than a 
photograph. However, because qualitative research involves fewer 
people, it is more difficult to say how much the results can be gener- 
alized to people other than the ones who were interviewed. 

Quantitative researchers make an effort to be objective, while in 
qualitative research, objectivity is not as important. In quantitative 
research, if subjective judgements are involved, the researcher tries to 
make sure that all of the judgements are made according to the same 
standard. Also, the researcher might have two people make the rat- 
ings and compare their ratings. For example, a researcher studying 
group discussion might include a rating on a scale of 1 to 5 of how 
good or poor the group discussion leader was. For a group leader to 
be rated 1, he/she might “allow one or two group members to domi- 
nate the discussion” and “frequently allow the group discussion to get 
off the topic.” For a group leader to be rated 5, he/she might “make 
certain all the group members participate” and “make certain the group 
discussion stays on the topic.” Descriptions like these help different 
raters use the same standard and makes the ratings more objective. 

Because quantitative research uses statistics, it is important to use 
a large number of participants in the study. In contrast, qualitative 
research generally uses relatively few participants — sometimes only 
one — and looks at them in a deeper way. 

Finally, quantitative research tends to involve some comparison, 
while qualitative may not. Researchers who take this approach might 
compare two groups of people or compare the same group of people 
at different times. For example, a quantitative researcher might com- 
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pare the English language test scores of Japanese and Chinese stu- 
dents, or he/she might compare the English language test scores of a 
group of Japanese students at the beginning and end of their first year 
of college. Qualitative researchers, on the other hand, sometimes just 
describe the characteristics of one group, without reference to another 
group. 

Quantitative and qualitative researchers can be very critical of one 
another. Quantitative researchers sometimes say that qualitative re- 
search is not real science. Qualitative research just looks at one per- 
son or one group of people, and it is not clear how the findings apply 
to other people or groups. Qualitative researchers, on the other hand, 
often say that quantitative researchers only look at the surface of what 
they are studying. Because they work with numbers, they cannot look 
at deeper meanings, and they cannot look from the participants’ view- 
point. 



Conclusion 

Qualitative and quantitative research are two approaches to doing 
research. They are each appropriate for different purposes and may 
even be combined. For example, researchers might use a qualitative 
approach first to identify some of the interesting concepts. They then 
might use a quantitative approach to take a closer look at those con- 
cepts. 

Application Questions 

Decide on a research question and write a brief sample of the kinds 
of questions you would write for a quantitative study and a qualitative 
study. 

For example, if your research question is “How are Japanese and 
Americans different in questions they ask when they first meet some- 
one?” If you are using a qualitative approach, you might ask Japanese 
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and Americans questions like these: 

“What kinds of questions do you ask when you first meet someone?” 
“What questions are too personal to ask the first time you meet some- 
one?” 

If you are using a quantitative approach, you might use questions like 
this: 

Rate the questions on whether you would be likely or unlikely to ask 
them the first time you met a new person. 

very likely very unlikely 

Are you married? 1 2 3 4 5 

When are you going to get married? 1 2 3 4 5 

Discussion Questions 

Read the paper in Appendix III. Does it use qualitative research, 
quantitative research, or both? Give examples of the approach(es) that 
you find. 
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In order to talk about research, you need to understand certain terms 
and expressions that researchers use. These include variables (includ- 
ing independent variables, dependent variables), and hypotheses. 

Variables 

The most basic part of a quantitative study is its variables. A vari- 
able is something that can be different in different people or in differ- 
ent circumstances. That is, it is something that can vary. For example, 
in some studies, sex — that is, whether a person is a male or female 
— is a variable. Those studies look at the differences between men 
and women. 

In a study of persuasion, for example, researchers might look at 
whether men and women find different kinds of arguments effective. 
In that case, sex is a variable. English proficiency can be a variable as 
well. It is measured in different ways. For example, a score on a test 
like the TOEFL, TOEIC, or Eiken can be used as a measure of Eng- 
lish proficiency. Some examples of variables are income, nationality, 
and IQ scores. Variables are important, because in quantitative re- 
search, they are what the researchers measure. Researchers in quanti- 
tative studies look at how variables are different in different groups, 
or how variables change over time, or how a change in circumstances 
can change the variable. 
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Some variables have only a limited number of values, while others 
have a range of values. Sex has to be either male or female — it has 
only two values. A single study would probably only use a few na- 
tionalities, say, English, American, Australian, French, and German. 
A test of English proficiency, however, might have values ranging 
from zero points to 100 points. Even for values with a range, how- 
ever, the researcher can specify a limited number of values. For ex- 
ample, if an English proficiency test has 100 points, the researcher 
might specify that participants with scores from 0-33 points have low 
proficiency, 34-66 points have intermediate proficiency, and 67-100 
points have high proficiency. 

Variables are basic to quantitative research. What a researcher wants 
to know is why a variable is different in different people. For ex- 
ample, why do some students learn English better than other students. 

Hypotheses 

Variables are used to form hypotheses. A hypothesis states the re- 
lationship between two or more variables. For example, the following 
are hypotheses: 

1. “Americans find rational arguments more persuasive, while Japa- 
nese find emotional arguments more persuasive.” In this hypothesis, 
nationality (Japanese or American) is one variable, and the type or 
argument (emotional or rational) is the other. 

2. “Friends in Internet relationships exchange more personal in- 
formation than friends in face-to-face relationships.” In this hypoth- 
esis, type of friendship (internet friendship vs. face-to-face friend- 
ship) is one variable, and amount of personal information exchanged 
is the other. 

3. “A woman wearing a suit will be judged as more intelligent than 
one who is wearing jeans and a t-shirt.” Judgments about intelligence 
are one variable, and type of clothing is the other. 
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The purpose of a research study is to test hypotheses or answer 
research questions. You make a hypothesis based on what is known 
about the subject from previous research. You can also make a hy- 
pothesis based on what seems logical. In some cases, however, you 
do not have enough information to make a hypothesis. In that case 
you make a research question, for example, “How is the amount of 
personal information exchanged in Internet and personal relationships 
different?” or “How do people judge a woman wearing a suit and a 
woman wearing a t-shirt differently?” 

Dependent and Independent Variables 

An important difference among variables is between independent 
and dependent variables. Independent variables are variables that, 
according to the hypothesis, are a cause of, or have an influence on, 
another variable. Dependent variables are variables that, according to 
the hypothesis, are an effect of, or are affected by, another variable. 

Variables are not inherently dependent or independent, that is, most 
variables can be either dependent or independent, depending on how 
they are linked to other variables in a hypothesis. For example, look 
at the following hypotheses; 

1. “Females reveal more personal information about themselves 
than males.” In this hypothesis, sex is the independent variable and 
how much personal information is revealed is the dependent variable. 

2. “People who reveal more personal information form closer 
friendships.” The independent variable is how much personal infor- 
mation is revealed, and closeness of friendships is the dependent vari- 
able. 

In the first case, the amount of personal information revealed is a 
dependent variable, and in the second case, the amount of personal 
information revealed is the independent variable. Therefore, a par- 
ticular variable can be either independent or dependent, depending on 
the hypothesis. 
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Look back at the three examples of hypotheses in the previous 
section. Which variables are dependent and which are independent? 

Control Variables 

Another type of variable is called a control variable. Control vari- 
ables are variables that the researcher is not studying, but which might 
affect the dependent variable. Control variables must be kept the same, 
or controlled, since they may have an effect on the dependent vari- 
able. For example, if the hypothesis states that students with higher 
language proficiency scores will attain a higher grade point average 
(GPA), it might be necessary to control for intelligence, since intelli- 
gence might also influence GPA. 

Control variables are used in different ways. One way is to use 
statistical methods to compare the language proficiency scores and 
GPA of students with the same IQ scores. Another way is to choose 
only one value for the variable. For example, if you think men and 
women will be different, you might want to do your study on only 
women. 

When you read a research p^per, you should consider whether the 
research took other variables into account, and whether there were 
other variables that might have had an influence. 

Conclusion 

Among the important concepts in research are variables and hy- 
potheses. A variable is anything that can vary; a hypothesis is a state- 
ment of the relationship between variables. When variables are put 
together into a hypothesis, one of the variables is independent, and 
the other is dependent. The dependent variable is caused or influ- 
enced by the independent variable. 

Application Questions 

Write three hypotheses. What are their dependent and independent 
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variables? Can you think of any variables that might need to be con- 
trolled in studies testing these hypotheses? 

Discussion Questions 

Read the paper in Appendix III. What variables did the researcher 
consider? What hypotheses did she test, or what research questions 
did she try to answer? 
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In quantitative research, one of the important issues is the relation- 
ship between the actual characteristic the researchers are looking at 
and how it is measured. For example, English proficiency is not some- 
thing we can observe directly. It must be measured somehow. But 
there are many ways of measuring English proficiency. Researchers 
can use the TOEFL, the TOEIC, the Eiken, or can write a test of their 
own. How is it best to measure English proficiency? In this chapter, 
we will look at some of the terms related to this issue. 

Differences Between Constructs and Variables 

In considering research, it is important to understand the relation- 
ship between a construct and a variable. A variable is essentially what 
we can observe or measure of a characteristic or ability, while a con- 
struct is the underlying characteristic or ability itself. Some constructs, 
like a person’s height or weight, can be measured directly, and there 
are established ways of measuring them. However, some constructs, 
like the closeness of a friendship or an attitude toward another cul- 
ture, cannot be observed directly. A researcher must measure close- 
ness of friendship indirectly using various measurement instruments, 
such as questionnaires, interviews, or observations, and the results of 
these instruments are variables. 

The variable for the construct of closeness of friendship, for ex- 
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ample, might be a questionnaire in which participants agree or dis- 
agree with these and other statements: 

1 . I spend as much time with my friend as possible. 

2. If my friend needed $100, and I had the money, I would give it 
to her. 

3. I can trust my friend with a secret. 

It is important to keep in mind, however, that though the results of 
such a questionnaire are intended to reflect the closeness of a friend- 
ship, the results are not themselves the closeness of a friendship. In 
doing or evaluating research, you must think about how well the meas- 
ures used for the variables actually reflect the underlying construct 
they are intended to reflect. 

Operationalization 

Therefore, an important issue related to the difference between a 
construct and a variable is operationalization. Operationalization is 
the link between the construct and the way it is measured; the opera- 
tional definition of a variable is the way that it is measured or quanti- 
fied. For example, English proficiency might be operationalized as a 
TOEFL score, a rating of a person’s ability to carry on a conversation 
in English under certain conditions, or the results of a reading test. 

Requirements of an Operational Definition 

There are two requirements for an operational definition. 

1 . It must be conceptually linked to the construct that it is intended 
to represent, that is, it must measure the characteristics that the con- 
cept actually has. In the questions about the closeness of a friendship 
above, the three questions could be intended to measure desire to spend 
time together, willingness to sacrifice for the friend, and trust. It is 
assumed that these are some of the characteristics that a close friend- 
ship would have. In measuring closeness of friendships, you should 
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think about the characteristics of friendship and ask questions about 
those characteristics. 

In addition, the questions should not measure any characteristics 
that should not be included in the construct. For example, the ques- 
tion, “My friend is rich” would not be an appropriate as part of the 
above questionnaire, if the amount of money the friend has is not a 
characteristic of closeness of friendship. 

If you are measuring English proficiency, you need to think about 
what kind of proficiency you want to measure. Imagine that your hy- 
pothesis is that the higher speaking proficiency is, the better chance a 
nonnative speaker has of forming friendships with Americans. For 
the purpose of your research, you need to measure speaking. What is 
the best way of measuring speaking? You could ask subjects of the 
research to prepare a speech. But that type of speaking is not related 

a 

to forming friendships. In this case, a better way to measure speaking 
ability would be to have the subject carry on a conversation and rate 
speaking ability based on that conversation. 

2. Secondly, an operational definition must be unique to that con- 
cept, that is, it must measure that concept and no other. For example, 
a test of listening proficiency that requires some reading of instruc- 
tions, questions, etc., may also be measuring reading proficiency as 
well as listening proficiency. That is, the score for listening profi- 
ciency may be influenced by the participant’s ability to read. In that 
case, the operational definition includes a concept that the researcher 
did not intend to measure. 

The Importance of Operational Definitions 

Operational definitions are important for three reasons. A clear 
operational definition allows researchers to be consistent throughout 
the research; it allows readers of the research to clearly understand 
the findings; and it allows other researchers to do the same research to 
see if they get the same results. 
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Clear definitions of constructs are important. In considering the 
relationship between variables and constructs when you read a re- 
search paper or do research yourself, it is necessary to consider the 
definitions that the researcher uses for the construct, if they are speci- 
fied. Whether a variable truly reflects a construct is, in part, depen- 
dent on how that construct is defined. For example, if the definition of 
English proficiency is related to fluency in speaking English, then a 
TOEFL score is not a good operationalization, because the TOEFL 
emphasizes a knowledge of grammar, reading comprehension, and 
other factors that are not be related to the ability to fluency in speak- 
ing. 



Conclusion 

Social scientists study many phenomena that cannot be measured 
directly, as height and weight can be measured. The actual phenom- 
enon is called a construct; the results of the measurement of that con- 
struct is called a variable. One of the most important issues in social 
science is the relationship between constructs and variables, and 
whether a particular variable accurately represents a construct. 

Application Questions 

Look at the hypotheses you wrote for Chapter 5. How might you 
operationalize the constructs? If you are going to use a questionnaire 
or an interview, write a few questions. If you are going to do an obser- 
vation, describe exactly how you would do it. 

For example, if you were studying the hypothesis “Friends in 
Internet relationships exchange more personal information than friends 
in face-to-face relationships,” you might operationalize the variables 
this way: 

1. Internet friends: people who originally met on the Internet and 
who have never met face-to-face 
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2. face-to-face friends: people who originally met in a face-to-face 
situation 

3. personal information: information that the participants rate as 
being private 

You might decide to use a questionnaire. You could ask partici- 
pants who have Internet friends to think of three topics they have 
discussed with Internet friends in the past month and three topics they 
have discussed with face-to-face friends in the past month. Partici- 
pants could be asked to rate the topics as follows: 

very private 1 2 3 4 5 not at all private 

Discussion Questions 

Read the paper in Appendix HI. How does the researcher operation- 
alize the constmcts? 
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Before you start your research project, you must go through sev- 
eral steps and make a number of decisions. You must do library re- 
search, decide on your hypotheses or research questions, design your 
study, decide how to gather data (interview, questionnaire, or obser- 
vation), decide who to gather data from, etc. 

Preparing for the Study 
Doing Library Research 

Your research project should begin with library research, that is, 
reading about the topic you are interested in. You need to read books 
and papers that researchers and theorists have written. You can search 
for papers on a particular topic using various online search services 
(see http://ilc2.doshisha.ac.jp/users/kkitao/online/other/search.htm). 
You should also check with the librarian at your university library 
about what kinds of search resources your library has, either on comput- 
ers or in printed periodicals like Linguistics and Language Behavior 
Abstracts (LLBA). Another possible resource is the Educational Re- 
sources Information Center (ERIC). You can find information about it at 
http://ilc2.doshisha.ac.jP/users/kkitao/online/other/search.htm#eric. 

When you do library research, you can read about what theorists 
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have written about the subject you are interested in and what other 
researchers have studied. What is already known about the subject? 
What are other researchers interested in? What hypotheses have they 
tested? What would you like to learn more about? Are there variables 
that previous studies have not considered, which you believe might 
have an influence? 

Writing Your Hypotheses or Research Questions 

Based on what you learned about the area you are interested in, 
you can write a hypothesis or research questions. You might look at 
variables that have not been considered before, or you might use the 
same hypothesis that another researcher has tested, but you can test it 
on another group of people to see if the results are different. 

As mentioned in the previous chapter, if you have enough infor- 
mation about the subject, you write one or more hypotheses. If you do 
not have enough information, you write one or more research ques- 
tions. 

You should be as specific as you can about the hypothesis or re- 
search question. If it is too general, it is difficult to test. For example, 
rather than asking, “How do college students talk with their friends?” 
you should ask a more specific question, such as “How does self- 
disclosure between college-age friends change over the first month of 
their friendship?” 

You should write down your hypotheses or research questions and 
keep them in front of you. As you plan the research, you need to keep 
them in mind. Every aspect of your research — every question you 
ask, every part of your procedure — should be related to testing the 
hypotheses or finding the answers to the questions. 

Designing Your Research Project 

How you design your research project depends on what you are 
trying to learn from your hypothesis. A hypothesis might be about 
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how two different groups are different. For example, if your hypoth- 
esis is that women reveal more personal information when they first 
meet than men do, then you design a study comparing men and women. 
You can write a questionnaire or do interviews, or you can do an ob- 
servation where you put pairs of men and pairs of women together 
and record their conversations. 

In other cases, you might have as your hypothesis that a persua- 
sive message that uses both rational and emotional appeals is more 
effective than persuasive messages that include only rational or only 
emotional appeals. In that case, you would test the attitudes of your 
participants, present each of three groups of people with one of your 
three persuasive messages, and then test their attitudes again. You 
would compare the attitudes before and after the persuasive message 
and find out whether and how much their attitudes have been changed 
by each type of message. 

There are various problems that you need to avoid when you de- 
sign your research. In the chapters on threats to validity and faulty 
research design, you will learn more about these problems. 

Questionnaires vs. Interviews vs. Observations 

Once you have designed your research, you need to decide how 
you will collect the information you need. There are two basic ap- 
proaches for gathering information indirectly — you can do inter- 
views, or you can have participants fill out questionnaires. These 
methods are referred to as being indirect, because you are asking par- 
ticipants about their behavior or their opinions, rather than actually 
seeing what they do. You can also gather information directly by do- 
ing observations, that is, by actually seeing what people do in differ- 
ent situations. Each approach has advantages and disadvantages, and 
you should carefully consider how you can best gather the informa- 
tion that you are looking for. 

Questionnaires. The main advantage of questionnaires is that you 
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can use them to gather information in exactly the same way from a 
large number of people in a short time. However, questionnaires are 
not flexible. You can only gather the information you have asked for. 
If you do not get enough information from a particular subject, or if 
you do not understand an answer, you can probably not go back to 
that person and ask more questions. 

A greater disadvantage of questionnaires (except those that are for 
the purpose of measuring language proficiency) are that you are de- 
pending on what the participants tell you about their attitudes or be- 
havior, not on what you can see and observe for yourself. You are 
required to depend on the truthfulness and self-understanding of the 
participants, and on their willingness to tell you the truth. It is well 
known that people are often not aware of, or are reluctant to truthfully 
report, their own behavior. 

Interviews. Compared to questionnaires, interviews take much more 
time. Also, it is difficult to do the interviews in exactly the same way. 
If the interviewer asks questions in different ways to different people, 
it may change the ways they respond. If different people do the inter- 
views, you have to train them so that they do them in the same way. 
However, interviews are more flexible than questionnaires. If you do 
not get enough information from a particular subject, you can ask 
more questions. If one subject says something interesting, you can 
get more information from him/her. 

As with questionnaires, when you do interviews, you are depen- 
dent on the participants to tell you the tmth. You cannot directly ob- 
serve what they do; you only know what they say they do. 

Observation. To overcome the disadvantage of having to believe 
what participants say, you might try to do direct observations. How- 
ever, observations also have disadvantages. One problem is that some 
of the constructs you are interested in might be impossible to observe. 
For example, if you want to measure the participants’ attitude toward 
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the death penalty, you probably cannot plan any observation that tells 
you what their attitude is. You need to get the person to express their 
attitudes, to ask questions, either by an interview or questionnaire. 

In addition, there are some activities that participants might not 
want you to observe. If they do allow you to observe them, partici- 
pants may behave differently, knowing that they are being observed. 
For example, if you are interested in self-disclosure, you might want 
to observe how two close friends talk to each other about personal 
problems. However, the friends might be reluctant to allow you to 
listen to such a conversation, or, if they did allow it, might be reluc- 
tant to talk as they would normally talk. 

Finally, it is difficult to observe behavior in a way that is consis- 
tent for different participants. For example, you may be interested in 
observing how close people stand when they talk to other people. You 
will find it difficult, in a natural situation, to control such differences 
as the relationships between the people who are talking (strangers, 
acquaintances, and friends) and the type of subject that they are talk- 
ing about. It might be possible for you to try to control the situation, 
for example, by choosing two strangers and giving them a particular 
topic to talk about. 

The following chart shows some of the similarities and differences 
among these three methods of gathering information. 



7. Getting Started and Choosing Your Approach 45 



Questionnaire Interview Observation 

Direct way x x o 

to gather info. 

Can gather 

large amount o x/o x 

of info, quickly 

Flexible x o x/o 

Consistent o x/o x/o 

Depends on 

willingness o ox 

to tell truth 

o = yes 
X = no 
x/o = maybe 

When you read a research paper, you should consider how the re- 
searcher chose to gather information, and what the problems may have 
been with that method. 

Choosing Participants 

As mentioned in Chapter 3, a population is all of the people to 
whom your study can be generalized; a sample is the people that you 
actually study. The ideal in quantitative research is to choose mem- 
bers of the sample at random. However, this is not often possible, and 
you should be aware of the limitations of the ways that you choose 
your participants. For example, suppose you are a student at Kyoto 
University, and you are doing a study where your population is Japa- 
nese university students. If you only use students from your own uni- 
versity, they may be different from other Japanese university students, 
so it may be difficult to generalize from your sample to the popula- 
tion. You should try to include other university students in your study. 
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As a practical matter, you can probably not choose your sample 
totally at random. However, you should try not to limit your sample 
to a particular group. For example, if you are doing an interview, do 
not just ask your friends to be interviewed. Your friends might be 
different from the population in general. Instead, try to choose a vari- 
ety of people to be interviewed. Ask students in the library or the 
cafeteria to be interviewed, rather than just asking people you know. 

In general, when you are doing quantitative research, the more 
participants you can use, the better. The more participants you can 
include in your study, the more likely it is that your sample will repre- 
sent the population as a whole. If you have a small number of partici- 
pants, a few participants who are not typical can have a big influence 
on the outcome of the study. However, if you have a large number of 
participants, some participants who are not typical will not have a 
great influence on the results. 

When you write your paper, you should discuss how you chose the 
participants and how that might have influenced your study. Think 
about how the sample you chose might be different from the popula- 
tion as a whole. 

When you read a research paper, you should consider how the re- 
searcher chose the participants and what effect that might have had 
on the results of the research. 

Conclusion 

Once you decide what you would like to research, you do library 
research and design a study to test the hypotheses or answer the re- 
search questions that you have formulated. You also decide how you 
will gather the information, depending on what kind of information 
you want and what your practical limitations are. You also choose the 
participants in your study, as much as possible, to be typical of your 
population as a whole. 
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Application Questions 

Write three related hypotheses that you would like to test or re- 
search questions that you would like to answer. (If you are carrying 
out a study as you read this book, you will do a study using these 
hypotheses or research questions. If you are not doing a study, you 
will use these to answer Application Questions in the rest of the Quan- 
titative Research section.) How would you test the hypotheses or try 
to answer the research questions? How would you gather informa- 
tion? How would you choose the participants for your study? 

Discussion Questions 

Read the paper in Appendix III. What method(s) did the researcher 
choose for gathering data? What advantages and disadvantages did 
that method have? 
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Questionnaires have the advantages that they can be used to gather 
information from a large number of people in a short time in the same 
way. However, a good questionnaire is not necessarily easy to write. 

Types of Questions 

There are various types of questions you can use in questionnaires. 
Which type you choose depends on the subject you are studying and 
what type of response you are trying to get. Whichever type of ques- 
tion you use, you should keep your research questions or hypotheses 
in mind. Be certain that each question is related to what you are trying 
to learn. 

Traditional T^pes of Questions 

You are probably familiar with some types of questions. The fol- 
lowing are some common ones: 

1 . true/false questions (a statement which the participant decides 
is true or false) 

2. short answer questions (a question that can be answered briefly) 

3. multiple choice questions (a question where the participant is 
given several possible answer to choose from) 

4. essay questions (questions that require the participant to write a 
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long answer) 

5. fill-in-the-blank questions (sentences with blanks for the par- 
ticipants to fill in) 

One common use of these types of questions is to try to find out 
how much subjects know about a particular topic. For example, if one 
of your variables is knowledge of American culture, then you can use 
these types of questions. 

You can also use multiple-choice questions when where you want 
to limit the number of possible answers. For example: 

Imagine that you told your friend Sarah a secret, and you found out 
later that she told someone else. What would you do? 

a. Get angry with her, but forgive her later 

b. Get angry with her, and end the relationship 

c. Get angry with myself for having trusted her 

d. Never speak to her again 

The type of question is quick and easy to score, but the problem is 
that there may be other alternatives. One possibility is to include a 
blank for another alternative: 

e. other (please specify. ) 

True/false questions and multiple-choice questions are easy to score, 
and subjects can answer a lot of questions quickly. However, it can be 
very difficult to write good questions. Short answer and fill-in-the- 
blank questions take longer to answer and to score. You have to make 
judgements about whether an answer is right or wrong, and you may 
have to make a system of categories to classify answers. However, 
they give subjects more flexibility in how they answer. Essay ques- 
tions are particularly difficult to score, because they give participants 
so much flexibility in their answers that they may be difficult to clas- 
sify. 
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Multiple choice questions. If you are using multiple-choice ques- 
tions to test comprehension or knowledge, they are more difficult to 
write than they may first appear. Some participants can figure out 
how to answer them not because they know the correct answer but 
because they can figure out which answer is the most likely. For ex- 
ample, the following question is based on a reading passage. 

Jane did not like her bedroom, because it was too . 

a. large 

b. small 

c. dark 

d. noisy 

Without even seeing the reading passage this is based on, you could 
probably guess that the answer is either a or b. Writers of multiple 
choice questions often use the opposite of the correct answer for one 
of the other alternatives. Good test-takers realize this, so they would 
be likely to guess a or b, even if they did not know the correct answer. 
Therefore you should avoid using the opposite of the correct answer. 

The following are some guidelines that will help you avoid prob- 
lems. 

1. Do not use an alternative that is the opposite of the right answer 
(because when there are opposites, one of them is often right). 

2. Do not use two incorrect alternatives that have similar mean- 
ings (because they are often both wrong). 

3. The alternatives should all be about the same length (because 
alternatives that are much longer or much shorter than the others tend 
to be right). 

4. Make sure that all of the alternatives are easily understandable 
(because the person answering the questionnaire cannot choose the 
correct answer if he/she cannot understand it). 
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As a general rule, the correct alternatives should be clearly correct, 
though they should not stand out; the incorrect alternatives should be 
clearly incorrect, but they should seem as if they could be correct. 
This is what makes writing multiple choice questions particularly dif- 
ficult. 

Tme/False questions. Writing trae/false questions is similar to writ- 
ing correct and incorrect alternatives for multiple-choice questions. 
True statements should be clearly true. False statements should be 
clearly false, but they should sound reasonable. 

Essay questions. Essay questions are more commonly used in quali- 
tative research, but you can also sometimes use them in quantitative 
research. You can use them to ask participants about their opinions or 
experiences. If you are doing a quantitative study, you form catego- 
ries for the responses and count the responses in each category. You 
might use essay questions if you are not sure what categories there 
might be. For example, if you are doing a study of communication 
within an organization, you might ask essay questions about who par- 
ticipants communicate with and what they communicate about. You 
could then divide the answers into categories according to whether 
the communication was with people who were higher, equal or lower 
in the organization and what the content of the communication was. 

Short answer questions. Using short answer questions in quanti- 
tative research requires you to make categories for the answers. You 
should be certain that the kind of answer you want is clear. 

FilUin-the-blank questions. Fill-in-the-blank questions can also 
be difficult to write, because it may not be clear what information you 
are looking for. When you write this type of question, you should be 
certain that it is clear what you want the respondent to write in the 
blank. Pre-testing (see below) is helpful to make certain your inten- 
tion is clear. 
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Scales 



Two more types of questions are used to get at the attitudes or 
opinions of subjects. One is a Lickeit scale, and the other is a seman- 
tic differential. 

Lickert scale. A Lickeit scale makes use of a list of statements. 
The subject indicates whether he/she agrees or disagrees with the state- 
ment. For example: 

If you strongly agree with the statement, circle 5. If you agree, circle 
4. If you are neutral, circle 3. If you disagree, circle 2. If you strongly 
disagree, circle 1 . 



1. In my organization, I communicate with 1 2 3 4 5 

people above me more than with people 

below me. 

2. My communication with people at the 1 2 3 4 5 

same level with me in my organization 

is mostly social. 

The subjects circle the number that indicates how strongly they 
agree or disagree. 

In some cases, you may want to ask more than one question about 
the same construct. For example, you might have another statement 
that says, “I mostly talk about social topics with people at the same 
level with me,” which would be similar in meaning to No. 2. This is a 
way of checking to make certain that you are getting an accurate re- 
sponse. (When you do this, you should try not to put the two ques- 
tions close together, so their purpose is not obvious.) 

The researcher averages the responses to questions about the same 
construct and compare different constructs or different groups. 

Semantic differential. In a semantic differential, the subjects are 



strongly 

disagree 



strongly 

agree 
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given two opposite nouns or adjectives, and subjects choose from a 
range of numbers. For example: 

Look at the picture of the woman and circle the number that repre- 



sents your impression of her. 








1 think the woman is: 








intelligent 


1 2 3 


4 


5 


unintelligent 


hard-working 


1 2 3 


4 


5 


lazy 


friendly 


1 2 3 


4 


5 


cold 


attractive 


1 2 3 


4 


5 


unattractive 



If you were using a semantic differential, you would average the 
responses for each item and compare different groups. 

Pre-Testing 

You should always pre-test any questionnaire that you write, that 
is, you should have a few people answer it and analyze their answers 
to make sure that they understand what to do, and that you are getting 
the kind of answers that you want. Because you may think that there 
is no problem with your questionnaire, you may feel that it is not 
necessary to do a pre-test, but this step is very important, so do not 
skip it. If you give your questionnaire to a hundred people, and then 
find that there is a serious problem, it will be too late. 

If possible, the people who pre-test the questionnaire should be 
from the same group of people who will eventually answer your ques- 
tionnaire, and they should fill out the questionnaire under the same 
conditions. For example, if the people who answer the questionnaire 
will be high school students, do not ask adults to answer it for your 
pre-test. Also, do not ask people to just look over the questionnaire to 
see if they see any problems. Have them actually answer the ques- 
tions, as you will expect your participants to do. If you have a chance. 
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in addition to having them fill out the questionnaire, ask the respon- 
dents whether they had any problems responding to the questionnaire. 

As you analyze the results, there are several things you should 
consider. One issue is how long it takes to fill out the questionnaire. If 
it takes too long, you might have difficulty getting participants to agree 
to fill out the questionnaire. In addition, participants might get tired 
of filling out the questionnaire, and they might not be careful in their 
answers, especially in answering the latter questions. 

It is also very important to make sure that the participants under- 
stand the questions. Do their responses, or their answers to your ques- 
tions after they have taken the questionnaire, indicate that there are 
items that they do not understand? In the case of multiple choice ques- 
tions, you should look at the wrong answers. If there is one wrong 
answer that is chosen by many of the people taking the pretest, you 
should try to find out why. Is that alternative correct under some cir- 
cumstances? Is there an interpretation of the question or the response 
that you did not consider? 

In the case of short answer or essay questions, you should look at 
the kind of responses you are getting. If you are getting only very 
brief answers when you expect long answers, you might re- word the 
question so that the participants give more information, for example, 
by asking for reasons for opinions. 

Conclusion 

Questionnaires are very useful for gathering information from a 
large number of people. In addition, there are a variety of different 
types of questions you can use, depending on what kind of informa- 
tion you are gathering. However, it is difficult to write a good ques- 
tionnaire. It is therefore very important to pretest the questionnaire to 
find any problems before you begin to gather data. 
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Application Questions 

Look at the hypotheses or research questions that you wrote for 
Chapter 7. Write true/false, fill-in-the-blank, short answer, multiple 
choice and/or essay questions that might be used to test your hypoth- 
eses or answer your research questions. 

Write examples of a Lickert scale and a semantic differential with 
at least four items each. 

Have two people answer your questions, and consider their an- 
swers. Did you get the kind of answers you expected? Do you need to 
revise your questions? 

Discussion Questions 

Read the paper in Appendix III. What kind of questions does the 
researcher use? How might the questionnaire have been improved? 
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Doing Interviews 
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One of the ways to get information for your research is to inter- 
view people. You use interviews to get opinions on possible solutions 
for a problem, learn about people’s experiences in and feelings about 
relationships, find out their reactions to a political event, etc. Know- 
ing how to do interviews effectively is an important skill. 

Preparing for the Interview 

When you interview someone, you ask questions with a particular 
purpose in mind. If you hope to achieve your purpose, your prepara- 
tion is almost as important as the interview itself. There are three steps 
that you should go through to prepare for an interview. 

1) Determine what specific information is needed. In the case of do- 
ing interviews to gather information for a research project, this 
information is determined by your hypothesis or research ques- 
tion. Be as specific as possible about what you need to know. This 
allows you to do four things: a) decide who would be appropriate 
to interview, b) find out more about the subject from other sources, 
so you can ask intelligent questions, c) plan the specific questions 
and the overall strategy, and d) assess when you have achieved 
your goals. 
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2) Select the interviewee(s). Who you interview is determined to some 
extent by step 1 . Given what you want to know, who can give you 
that information? In the case of doing quantitative research, you 
choose your interviewees from the group of people that you are 
studying. That is, if you are studying the interpersonal relation- 
ships of Japanese college students, you would interview Japanese 
college students. 

3) Prepare questions. Based on whom you are interviewing and what 
information you want to get from them, make a list of the ques- 
tions you want to ask. You should realize that you will not neces- 
sarily follow your list of questions exactly, but you should have a 
basic list of questions that will help you achieve your purposes. 

Types of Questions 

Types of questions you should use. There are four types of ques- 
tions that you can use in interviews. Some are questions that you write 
in advance for the interview; others are ones that you will ask in re- 
sponse to your interviewee’s answers. As you write your questions, 
you should think about whether the kind of questions you are writing 
will get the kinds of responses that you want. Both types are impor- 
tant. 

1) Closed-ended questions. These are questions requiring a direct re- 
sponse, a “yes” or “no” answer or one or two sentences. They have 
clear “right” answers. These are related to a fact rather than an 
opinion, speculation, etc. “How old are you?” “Have you ever been 
overseas?” and “How and when did you and your best Mend meet?” 
are examples of closed-ended questions. 

2) Open-ended questions. These are questions that do not have a def- 
inite answer and that allow the interviewee to answer in any direc- 
tion. These may ask for an opinion, ask the interviewee to specu- 
late about something, etc. Examples of open-ended questions in- 
clude, “How do you think Internet friendships are different from 
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face-to-face friendships?” “How do you think getting married will 
affect your career?” and “Do you think that your small group makes 
decisions more effectively when there is a strong leader? Why or 
why not?” 

3) Mirror questions. These questions are used to reflect back to the 
interviewee what you believe she/he said in order to check to make 
sure you understood correctly. It might also be used to get her/him 
to elaborate on or clarify a point. A mirror question might use an 
expression like, “If I understand what you are saying, you agree 
with the school’s new policy, because it will allow students to take 
a wider variety of classes.” This type of question both allows you 
to check your comprehension and gives you a chance to show the 
interviewee that you are listening carefully and are interested in 
what he/she is saying. 

4) Probing questions. These are questions used to get an interviewee 
to elaborate on or clarify a response. You use these when you do 
not understand what the interviewee meant. You also use them when 
the interviewee says something unexpected or interesting and you 
want to learn more. For example, you might ask, “Why do you 
say...?” or “What do you mean when you say...?” You cannot plan 
these question in advance, but if you use them well as you are 
interviewing, you will often get interesting information. 

As you are writing your questions, you should keep your hypoth- 
eses or research questions in front of you, and you should compare 
each question you write with them. Will that question help you an- 
swer the question or test the hypothesis? If the answer is “no,” then 
you should not ask the question. 

Types of questions you should avoid. When you are writing ques- 
tions for your interview, there are also two types of questions that you 
should not ask. They will not result in good responses, and they may 
make the interviewee distrust you. 
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1) Leading questions. These are questions worded in a way that en- 
courages a particular answer. For example, in English, negative 
questions about an opinion encourage a positive answer. The ques- 
tion “Don’t you think that face-to-face friendships are more “real” 
than Internet friendships?” prompts the interviewee to answer “yes.” 
This type of question should be avoided. The purpose of your in- 
terview is to find out about the interviewee and his/her opinions, 
not to urge the interviewee to give a certain response. 

2) Loaded questions. These are questions that assume something that 
may or may not be true. For example, “Why is it difficult for a 
small group to make a decision when it does not have a strong 
leader?” would be a loaded question. What it really means is, “It is 
difficult for a small group to make a decision when it does not 
have a strong leader. Why do you think this is true?” The assump- 
tion that the question is based on is that small groups do have dif- 
ficulty making decisions without a strong leader. Your interviewee 
may not agree with this assumption, and that makes it a difficult 
question to answer. It may also lead the interviewee to a specific 
type of answer, and again, that is not what you want. You want to 
know what the interviewee thinks and has experienced. 

Deciding the Order of the Questions 

As you prepare for the interview, you should also think about the 
order in which to ask your questions. One method is to start with 
broad questions and move to specific ones. Another is to start with 
specific questions and move to broad ones. You may need to adjust 
your questions according to the responses you get. If your interviewee 
is getting off the subject answering open-ended questions, you might 
need to switch to closed-ended questions. If the interviewee is an- 
swering only “yes” or “no” to closed-ended questions, you might 
switch to open-ended questions. 

However, if you are doing interviews for a quantitative research 
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project, you must be careful not to vary too much from the interview 
script. If the interviews are done too differently, it makes them diffi- 
cult to compare from a quantitative point of view. Therefore, if you 
are doing interviews for a quantitative study, you should make certain 
you are getting the information you are asking for, without getting too 
far from the main point. 

Preserving the Results of the Interview 

You also need to decide how you will preserve the results of the 
interview. In some cases, you may want to make an audio or video 
tape of the interview. This would preserve the information, and you 
could always go back and listen to it, but it might make the inter- 
viewee self-conscious. Also, equipment does sometimes malfunction, 
and video equipment in particular may be difficult to set up and handle. 

The alternative is to take notes. If this is what you decide to do, it 
is probably best to have a form to record the responses. If there are a 
limited number of possible responses, you can put them on the form 
and then just circle the one the interviewee indicates. If you ask, 
“Where did you meet your best friend?” you can put possible responses 
such as “at school,” “in my neighborhood,” “through a mutual friend.” 
Then you can just circle the interviewee’s response. Whatever you 
can do to make it easier to record the response, you should include on 
the form. For example, if one question is “How long have you known 

your best friend?” you can put “ years, months” on the 

form and just fill in the numbers. However, if you are asking ques- 
tions that may have long answers, or an unlimited variety of answers, 
it may be difficult for you to write down everything. 

Pre-Testing 

As with questionnaires, you should pre-test the interview script. 
Try it out by actually interviewing the kind of people for whom the 
interview is intended. Find out whether the interviewees understand 
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the questions and whether you are getting the kinds of responses you 
need. This will also help you refine the response form, if you are 
using one, so that you can make it more effective by finding out what 
types of responses you actually get. 

Carrying Out the Interview 

Once you are ready to start interviewing, you need to go through 
several steps. 

1) Arrange for the interview. Identify yourself and the purpose of the 
interview. Try to do the interview at a time and place that is con- 
venient for the interviewee. 

2) Opening the interview. Before you start the interview itself, try to 
put the interviewee at ease, if necessary, and build rapport. In a 
general way, you can explain what the interview is about, though 
you do not want to give too much information, since that might 
influence the interviewee’s responses. For example, you might say, 
“I’m doing interviews to learn about how people communicate with 
co-workers.” You might also thank the interviewee for taking the 
time to be interviewed. 

3) Doing the interview. Ask the questions that you have prepared but 
adjust the questions according to the responses that you are get- 
ting. Use probing questions and mirror questions to follow up on 
answers given during the interview. Throughout the interview, you 
should evaluate the answers you are getting. Do you understand 
them? If not, ask the interviewee to repeat the answer, or ask mir- 
ror questions. Are you getting the specific information you need? 
If not, ask probing questions. If the interviewee says something 
particularly interesting, use probing questions to get him/her to 
elaborate on that point. 

4) Closing the interview. You should not just stop the interview. You 
can finish by emphasizing important points or summarizing. You 
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should definitely thank the interviewee, and mention that the inter- 
view has been helpful. 

5) Evaluating the interview. After the interview, ask yourself whether 
you got the information you wanted. If not, why not? In any case, 
ask yourself how the interview could have been improved. 

6) Analyzing information. The last step is to synthesize the informa- 
tion into report or whatever your final product is. In the case of 
using an interview in a quantitative research study, you need to 
count the responses that fall into particular categories. 

Checklist 

As you are preparing for and doing an interview, you can use the 
following list. 

Preparing for the Interview 

1 . Decide what specific information you need. 

2. Decide who you will interview. 

3. Prepare your questions. 

a. Combine open-ended and closed-ended questions as appro- 
priate. 

b. Be certain that you are avoiding both leading and loaded 
questions. 

4. Decide on the order of your questions. 

5. Decide how you will record the results of the interview. 

6. Pre-test your interview script and adjust it as necessary. 

Doing the Interview 

1. Contact the interviewees and arrange for the interviews. 

2. Open the interview by putting the interviewee at ease, if neces- 
sary. 

3. Briefly explain the purpose of the interview. 

4. As you do the interview, evaluate the answers you are getting. 
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a. Adjust the order of the questions as necessary. 

b. Use mirror questions and probing questions as appropriate. 

5. Close the interview by summarizing the content or emphasiz- 
ing an important point. 

6. Express your appreciation to the interviewee. 

After the Interview 

1 . Evaluate the interview to decide if you got the information you 
wanted. 

2. Analyze the information from the interview. 

Conclusion 

Interviews have the advantage that they are more flexible than 
questionnaires, though it is more difficult to gather large amounts of 
information. To do successful interviews, the preparation is very im- 
portant. You need to write good questions. As you are carrying out the 
interview, you need to be flexible both about the order of the ques- 
tions and in asking mirror and probing questions. 

Application Questions 

Look again at the hypotheses or research questions that you wrote 
for Chapter 7. Write a few interview questions for those hypotheses 
or research questions. 

Make a form for answers to the interview questions that you wrote 
and interview two people. Use mirror and probing questions as ap- 
propriate. Based on the results of these interviews, analyze your ques- 
tions and see if they need to be rewritten. 

Discussion Questions 

Read the paper in Appendix III. How could the researcher have 
used interviews? 
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As mentioned in a previous chapter, interviews and questionnaires 
are an indirect way of finding out about how people communicate. 
However, when using questionnaires or interviews, researchers de- 
pend on participants to tell what they do. Participants may not be 
aware of what they do, or they may not be willing to tell the truth 
about what they do. In contrast, observation is a direct way to find out 
about the behavior of participants. 

Making Observations for Research 

Everyone observes the behavior of other people at one time or 
another. In these ordinary, daily observations, you notice who is good 
at expressing their ideas and who is not. You observe who talks a lot 
about themselves and who is a good listener. You may come to real- 
ize, for example, that if you give a friend personal information, they 
are likely to give you personal information in return. If you are rude 
to other people, they are rude in return. 

Researchers sometimes also observe communication behavior. 
However, their observations must be different from observation in 
everyday life. In general, their observations must be more organized 
than ordinary, daily observations. 
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Observing in Similar Conditions 

One way that researchers observe behaviors differently than people 
in everyday life is that researchers may make an effort to observe 
behaviors by different people under the same, or at least similar, con- 
ditions. If behavior is observed under different conditions, there might 
be different influences. Therefore, the researcher may try to create 
situations where the conditions are the same or similar for different 
people. Since refusals of requests may be influenced by a variety of 
factors such as the form of the request, the relationship between the 
two people, and the size of the request, it is difficult to compare them 
when they occur naturally. 

In addition, the behaviors may not occur often enough for the re- 
searcher to find enough examples of the behavior. If the researcher is 
studying refusing of requests, the researcher may not be able to find 
many that occur naturally. In addition, the behavior that the researcher 
is interested in may occur in private, so it is difficult for the researcher 
to observe. 

One way to solve these problems is to use a confederate. A confed- 
erate is a person who is helping the researcher and whose behavior or 
words are decided by the researcher. The subjects of the research may 
not know that the confederate is helping the researcher. For example, 
a confederate might make requests of various people and record the 
results. The confederate could make the same request in the same 
way to various people. Also, the researcher could choose a request 
that would be likely to be refused so that there would be many ex- 
amples of refusals. 

Another way to solve these problems is to use role play. Rather 
than gathering naturally occurring refusals, the researcher might ask 
pairs or groups of subjects to imagine they are in a certain situation 
and to say and do the things they would in that situation. For example. 
Subject A and Subject B might be told that they are classmates at a 
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university who do not know each other well. Subject A might be given 
instructions to ask to borrow Subject B’s dictionary, and Subject B 
might be given instructions to refuse. The two subjects then say what 
they think they would say in that situation. Role plays allow the re- 
searcher to have a lot of control over the situation and to gather many 
examples of the speech or behavior, but they may not be as realistic as 
naturally occurring situations. 

When you read a research paper, you should consider how the re- 
searcher did the observations and whether they were done in similar 
conditions. If they were not done in similar conditions, you should 
consider how that might have influenced the results. 

Making Lists of Behaviors 

Another way that researchers do observations in a more organized 
way is that they look for a list of general or specific behaviors. As a 
researcher, you cannot just watch what people do. You have to look 
for specific things. Before starting your observation, you make a list 
of the behaviors that you are looking for. For example, if you are 
studying gestures by observation, you have a list of gestures. Your list 
might include pointing, nodding for “yes,” shaking the head for “no,” 
waving for “hello,” waving for “goodbye,” and so on. Each time you 
saw the people you were observing making one of the gestures on 
your list, you would count that gesture, and record it on your list. 
Watching behaviors and counting them or rating them is called “cod- 



There are two major approaches to making a list of behaviors to 
look for. One approach is to make a list of very specific behaviors for 
the observer to look for. The researcher only looks for the exact be- 
haviors on the list and does not need to interpret the behaviors. For 
example, if the researcher is interested in exchanges of opinion in 
group discussions, a list of specific behaviors may have such items as 
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“Asks for an opinion from another group member” or “Agrees with 
the opinion of another group member.” If one of the group members 
says, “So what do you think, Joe?” or “I think so, too, Sarah,” the 
observer marks the appropriate items on the list. 

However, sometimes the list has more general behaviors, and the 
observer needs to make inferences about or judge how to interpret the 
behavior. For example, an item on a more general list might be, “Shows 
support for other group members’ expressions of opinion.” In that 
case, the behaviors that are to be counted might not be as clear, and 
the observer might have to do some interpretation. If a group member 
says, “Thank you for that contribution, Jean” or pats Len on the shoul- 
der after he expresses an opinion, then these might be interpreted as 
giving support for expressions of opinions. However, if a group mem- 
ber smiles at the person who expresses an opinion, should this be 
counted as showing support? The observer has to decide, and differ- 
ent observers might make different decisions. 

A researcher has to decide which kind of list — general or specific 
— to use. The more specific the list, the easier it is for observers to 
find the behaviors that are described on the list. A specific list also 
makes it easy for all of the observers to agree on what they see when 
they are observing. On the other hand, a specific list cannot possibly 
cover all of the possible behaviors. The research might try to list all of 
the behaviors that could be considered to be “showing support for 
other group members’ expressions of opinion,” but he/she would prob- 
ably not think of all of them. 

When you read a research paper, you should consider the list that 
the researcher used. Did it involve general or specific behaviors? Were 
the observers consistent in their observations? 

Using Rating Scales 

When you count behaviors, the behavior is either there or it is not. 
If you are counting gestures, you might find that the participant waves 
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good-bye once or five times or ten times. In addition to making obser- 
vations by counting behaviors, researchers can make observations 
using scales. When you use a scale, you rate the behavior according 
to how strong or intense it is. For example, if you are looking at a 
small group discussion, you may want to rate how attentive to the 
discussion each group member seems to be. Some of the group mem- 
bers may seem to be paying a lot of attention to the discussion, and 
others may seem to be paying little or no attention. For your observa- 
tions, you might make a scale to rate how involved each member of 
the group is. One way to make such a scale is to make a semantic 
differential. For example, you might have a scale like this for each of 
the group members: 

Group member A is: 

Not at all attentive 1 2 3 4 Very attentive 

You may also list the possibilities. 

Group member A is: 

1 Not at all attentive 

2 A little attentive 

3 Somewhat attentive 

4 Very attentive 

Another way to make a rating scale is to describe the behaviors at 
each level. For example: 

Group member A is: 

Not at all attentive. Leans back, does not make eye contact with 
other group members, does not contribute to the discussion. 

A little attentive. Spends most of the time leaning back, but occa- 
sionally leans fonvard, makes eye contact once in a while, makes 
rare contributions. 

Somewhat attentive. Makes some eye contact with other group mem- 
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bers, occasionally makes contributions. 

Very attentive. Leans fonA/ard, makes eye contact with other group 
members most of the time, makes many contributions to the dis- 
cussion. 

Rating scales are useful in some cases, but they are difficult to use, 
because different observers may interpret the behaviors in different 
ways. Observers need to be well trained, and even then, there might 
be problems. Some observers will always tend to rate too high; others 
will always tend to rate too low. In addition, if raters know and like or 
dislike the participants, they may rate them more or less strictly. In 
some cases, rating scales may be necessary or useful, but you need to 
train observers very carefully and compare their results to make cer- 
tain that they are reliable. (This is true of any system of observation, 
but it is especially true of scales.) 

Other Types of Observation 

Though when we think about observation, we generally think about 
watching or listening to what people do or say. However, it is also 
possible to “observe” written material. If you are studying a relation- 
ship carried on by e-mail, for example, you might use the e-mails 
themselves as the material that you observe. You might count the num- 
ber of times each correspondent discusses his/her feelings about the 
relationship. You can use students’ compositions and count the types 
of support they provide for their opinions in the compositions. 

Recording Observations 

When observing behaviors, the researcher needs a way to record 
the behaviors. The observer can take notes, record the words on audio 
tape, or record words and behaviors on video tape. 

If the observer takes notes or counts behaviors as they actually 
occur (as opposed to watching a video or listening to a recording), it 
may be difficult for him/her to both observe and take notes at the 
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same time. In addition, if the behavior is not recorded in any way, 
there is no way to check the observations to make sure that the count- 
ing is accurate. Because of this, taking notes is more common for 
qualitative research than for quantitative research. 

There are ethical problems with recording someone without their 
knowledge. In the US, it is considered unethical to record someone 
without their knowledge, and journals will not publish research pa- 
pers based on such recordings. One way to handle this is to elicit the 
behavior that you want to observe, record it, and then explain that you 
are doing a research project and ask permission to use the recording. 
If you are refused permission, you must erase the recording. If the 
researcher uses role plays, the participants can be told that the role 
play is being recorded. 

Doing a video recording has similar problems to doing an audio 
recording, plus some additional problems. However, if you are ob- 
serving behaviors as well as words, a video recording may be neces- 
sary. It is difficult to videotape in a natural situation, and, as with an 
audio recording, permission is necessary. In addition, participants may 
be self-conscious about being videotaped, though they may become 
accustomed to it. Also, it may be difficult to find the most useful angles. 
You need to be certain that you can see the things you need to be able 
to see. You may need to be certain, for example, that you can see the 
facial expressions of participants. 

To allow observers to count or rate behaviors easily, it is useful to 
make a form. The form has the lists of behaviors or rating scales, and 
the observers can count or rate the behaviors. You should have at least 
two observers counting or rating the behaviors, and you should com- 
pare their results. If their results are very different, you will need to do 
more training, make a clearer system of counting or rating, etc. 
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Conclusion 

Doing observations has the advantage of allowing the researcher 
to directly see what people do, rather than depend on what they say 
they do or would do. However, there are many limitations. The re- 
searcher may be interested in behaviors that people do not want to 
have observed. For the purpose of quantitative research, it is often 
difficult to do observations in similar ways. In addition, it is impor- 
tant to have an organized, consistent way to record and count the be- 
haviors that the researcher is interested in. 

Application Questions 

Using the hypotheses or research questions that you wrote for Chap- 
ter 7, design an observation that would test those hypotheses or an- 
swer those research questions. What would you observe? How would 
you try to make sure that you were observing different people in simi- 
lar situations? 

What behaviors would you count, and how would you count them? 

Would you use a rating scale? If so, what kind? What kind of form 
would you use to code the behaviors? 

How would you record the observations (note-taking, audio tape, 
video tape)? Why? 

Discussion Questions 

Read the paper in Appendix III. Did the researcher use observa- 
tion? If so, how? 
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The goal of research is testing hypotheses. To test hypotheses, two 
things are necessary — variance and control. Variance means having 
different values for the dependent variables. That is, imagine you are 
studying differences in judgments about the intelligence of women, 
depending on their clothing. You show people pictures of the same 
women wearing a suit in one picture and jeans and a t-shirt in another 
picture. Variance occurs if the ratings for a woman dressed in a suit 
are different from those for a woman dressed in jeans and t-shirt. 

There are three types of variance, and you need to make certain 
that any variance you find is not caused by errors or by lack of con- 
trol. Control is the ability to explain as much of the variance as pos- 
sible. The more variance there is that cannot be explained, the less 
control there has been in the study. Lack of control allows for rival 
explanations for the results. 

Types of Variance 

Variance can be divided into three types: systematic experimental 
variance, extraneous variance, and error variance. Systematic experi- 
mental variance is differences in dependent variables caused or influ- 
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enced by the independent variable. For example, if you are studying 
how people judge a woman wearing a suit and one wearing jeans and 
a t-shirt differently, you might show participants pictures of a woman 
dressed those types of clothing (but exactly the same in every other 
way) and ask them to rate her on a semantic differential as to whether 
she is friendly, serious, intelligent, etc. If you find that she is judged 
as more friendly when she is seen wearing jeans and a t-shirt, you 
assume that the difference is due to the way she is dressed. This is the 
type of variance that researchers want to maximize. 

Extraneous variance is caused by unspecified and unidentified vari- 
ables. Imagine that the woman is smiling in the picture with the t-shirt 
and jeans and has a serious expression in the picture with the suit. She 
might be judged as being more friendly in the picture where she is 
wearing a t-shirt and jeans. However, you would not know whether 
this was due to clothing or due to the expression on her face. This type 
of error should be minimized. When you design your research, you 
should try to be certain that there are no differences other than the 
ones you are studying. If you are studying the differences in judg- 
ments about a woman based on two pictures, you should make certain 
that, except for her clothing, the pictures are as similar as possible — 
the same expression, on her face, the same position, etc. Otherwise, 
you will not know why there are differences in the participants re- 
sponses. 

Error variance is variance caused by sloppy experimental proce- 
dures, errors in measurement, coding errors, key punching errors, etc. 
This type of variance should also be minimized. 

In quantitative research, the purpose of research design is to achieve 
control — to minimize the extraneous variance. Researchers must try, 
as much as possible, to either keep variables (other than the indepen- 
dent variables) that might influence the dependent variables the same 
or to measure them and control their influence statistically (control 
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variables). 



Threats to Validity 

Internal and External Validity 

Threats to validity of research are factors that may influence the 
outcome, other than the independent variables that you are studying. 
They are factors that might cause extraneous error. 

In general, threats to validity can be divided into two categories — 
threats to internal validity and threats to external validity. Internal 
validity is the degree to which we can attribute variance to the inde- 
pendent variable. That is, the better the internal validity, the more 
certain we can be that differences in the dependent variables were 
influenced by differences in the independent variable. This concept is 
related to control, discussed in the previous section, in that threats to 
internal validity are factors that reduce systematic experimental vari- 
ance and increase extraneous variance. In other words, threats to in- 
ternal validity cause the dependent variable to change for reasons that 
the researcher cannot explain or account for. If the woman is smiling 
in the picture where she is wearing the t-shirt and jeans and has a 
serious expression in the picture where she is wearing a suit, that is a 
threat to internal validity. 

External validity, on the other hand, is the extent to which the re- 
sults of the study can be generalized, that is, it’s relationship to situa- 
tions outside of the study itself. In a sense, it is how realistic the situ- 
ation used in the research is. 

Threats to External Validity 

In considering the external validity, the reader of a study must con- 
sider whether the conditions found in the study actually occur in the 
“real world.” For example, a study of the effectiveness of a language 
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teaching method may involve using the method only once or over a 
short period of time. Since, for the most part, language classes use a 
method over a much longer period of time — possibly several weeks 
or months — there have to be questions about the external validity of 
such a study. Also, in a study using pictures of a woman and making 
judgments about her characteristics might not have good external va- 
lidity. It is not often that people make judgements about a person based 
on pictures rather than face-to-face. 

The selection of participants in the study may also have an influ- 
ence on the extent to which the results of a study can be generalized. 
If the participants in a study are selected from non-native speakers 
studying English in the United States, it is difficult to generalize the 
findings to students of English who live in countries where English is 
a foreign language. 



Threats to Internal Validity 

Threats to internal validity can be divided into several categories 
according to the issues that they are related to. In this section, you 
will learn about threats to validity related to how research partici- 
pants are chosen and grouped, as well as their continued participa- 
tion, and how these influence the results of the study. 

Selection-Related Effects 

In quantitative research, it is ideal to select participants at random. 
This helps make sure that the members of the sample reflect the popu- 
lation that they are intended to represent. (The members of the sample 
are the actual participants in the study; the population is the group of 
people that the study is generalized to. For example, the sample may 
be a group of randomly chosen students at a Japanese university.) 
However, it is not always possible to randomly select participants, 
and so researchers often use other ways of choosing participants, in- 



76 Part II Quantitative Research 



eluding self-selection and using intact groups. Each of these repre- 
sents a threat to internal validity. 

Self-selection. Self-selection means that the participants themselves 
choose to participate in the study, that is, the common practice of 
having participants volunteer to join the study. Participants who vol- 
unteer for a study may be different from people who do not. This is 
particularly a problem when participants who volunteered are com- 
pared to participants who did not (for example, who participate in the 
study as part of a class). In a study of a language learning method, if 
volunteers are used to try the method in question, the volunteers are 
likely to be more motivated, be better at or more interested in learning 
language, and have higher language proficiency than non- volunteers. 
Therefore, you should avoid comparing volunteer and non- volunteer 
groups if you can, and you should be careful about generalizing from 
volunteers to the general population. While it is a good idea to avoid 
self- selection, it is not always practical — you are not always able to 
get randomly chosen participants to participate in a study. Therefore, 
if self- selection is used, you should describe the self- selection and its 
possible effects on the results. Also, you should take particular care in 
comparing volunteer and non- volunteer groups and generalizing from 
a volunteer sample to the general population. 

Intact groups. The use of intact groups is also a potential problem. 
Intact groups are groups that are already together before the research, 
such as students in a class or members of a club. There are two major 
problems with this practice. One problem is that the group may differ 
in some variable, since they have not been randomly assigned to the 
groups. One group might, on average, have higher language profi- 
ciency or might have spent more time in English-speaking countries. 
If so, the groups are not necessarily comparable, and differences be- 
tween them may be explained by differences in the two groups rather 
than the influence of the independent variables. Also, the different 
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intact groups may have already established ways of interacting among 
themselves, and this may influence the dependent variable as well. If 
you are studying how small groups make decisions, an intact group 
may already have a leader. Group members already know each other. 
These and other factors might affect how they make decisions and 
therefore influence the results of the study. 

You should avoid intact groups if possible and instead assign par- 
ticipants to groups randomly. However, in many cases, this is not pos- 
sible. In those cases, there are ways to reduce the problems. You can 
identify other variables that might influence the dependent variable, 
measure them, and use statistical procedures to determine whether 
they have influenced the outcome and to control them. For example, 
in studying the effectiveness of a language teaching method with two 
intact groups, the language proficiency of the two groups needs to be 
measured and controlled. 

Mortality. Another issue related to the participants is mortality. 
Mortality is the dropping out of participants from the study for any 
reason. This is a problem because the participants who drop out may 
be different on one or more variables than those who do not. In addi- 
tion, mortality can change the average of the group on one or more 
characteristics. You should specify how many participants dropped 
out of the study and why. If possible, you should try to find out whether 
these participants were different on any important variables from par- 
ticipants who did not drop out. 

Conclusion 

As you design your research, you should do it in a way that allows 
you to explain why the results for different groups are different. In 
order to do that, you control the people that you choose and the condi- 
tions under which you do the study. You try to maximize the external 
validity, that is, the degree to which your results can be generalized to 
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the “real world,” and the internal validity, the degree to which you 
can explain the differences among groups. As a reader of a research 
paper, you should consider what threats there might be to internal and 
external validity. 

Application Questions 

How can you maximize systematic experimental variance and mini- 
mize extraneous variance in your research project? For example, if 
you are testing the hypothesis that a woman wearing a t-shirt and 
jeans would be 

What are the problems with the external validity of your study? 
Who will be the participants of your study? How does that influence 
its external validity? 

How will you choose the participants for your study? Will you use 
intact groups or self- selected participants? If so, how can you avoid 
the problems with these? 

Discussion Questions 

Read the paper in Appendix III. What threats to internal and exter- 
nal validity are there? 
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In doing research, you need to consider factors that might allow 
different interpretations for your results. In the previous chapter, we 
discussed threats related to the way that participants are selected. Two 
other types of threat to internal validity are time-related effects and 
people-related effects. 



Time-Related Effects 

A category of threats to validity is time-related effects. These are 
related to the passage of time between the beginning and end of a 
study. 

History Effects 

One type of time-related threat to validity is a history effect. His- 
tory effects are the effects of incidents not related to the study that 
take place between the beginning and end of the study and which may 
affect the results of the study. For example, you might be studying the 
effects of persuasive messages on Americans’ attitudes on gun con- 
trol. You measure the attitudes of your sample toward gun control, 
have them listen to a persuasive message, and measure their attitudes 
again two weeks later. However, there might be a story in the news 
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related to gun control between the first measurement and the second 
measurement. If so, you will not know if the change in attitude was 
due to the news story or to the persuasive message. 

One way that you can find out about history effects is to talk to 
research participants about anything that might have influenced their 
opinion. You should also have a control group. The control group 
takes the pre-test and the posttest, but it does not have the same treat- 
ment (that is, in this case, the persuasive message). If the group that 
heard the persuasive message changes their attitudes and the group 
that did not hear the persuasive message does not, it is more likely 
that the persuasive message influenced or caused the change. 

In addition, you should have the second test as soon as possible 
after the first test. This will reduce the possibility that there will be a 
history effect in between the first and second test. (On the other hand, 
you might be interested in how long changes in attitude last, so you 
might have reasons to have a longer time between the first and second 
tests.) 

Maturation Effects 

Another type of time-related effect is a maturation effect. Matura- 
tion effects are normal changes in participants over time. These changes 
are probably most obvious in children, since their muscle control, 
their competence in their own language, etc., are still maturing, but 
maturation can be a factor in studies with adults, too. In a study done 
with non-native English speakers living in an English-speaking coun- 
try, the fact that participants are in an English-speaking environment 
will certainly influence their language proficiency, independent of the 
manipulation in the study. Again, having a control group and keeping 
the study short help limit the effects of maturation or at least allow 
researchers to recognize them. In addition, researchers may use two 
pretests, a treatment, and then a posttest, and then compare the two 
pretests to find out if there is maturation occurring. If there is no 
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difference between the two pretests, maturation is probably not a fac- 
tor. 



People-Related Effects 

Another category of threat to validity is people-related issues. This 
category is related to the attitudes of those involved in the study, both 
participants and researchers. 

The Hawthorne Effect 

One people-related effect is called the Hawthorne Effect. In some 
cases, just the fact of being involved in a study and/or the novelty of 
the treatment itself can cause participants to perform better than they 
normally would. Therefore, when researching the effectiveness of a 
new language teaching method, you have to take into account that the 
very newness of the new method might have an effect on the results, 
apart from the characteristics of the method itself. Carrying on the 
treatment for a longer period of time allows the participants to get 
accustomed to the new conditions. 

Participant Expectancy 

Another type of people-related effect is participant expectancy. If 
participants understand what the study is about or what you expect of 
them, they may consciously or unconsciously try to give you the re- 
sults that you are looking for. To the extent possible, subjects should 
be unaware of the purpose of the study and the hypotheses being tested. 

Researcher/Coder Expectancy 

Another type of people-related effect is researcher/coder expect- 
ancy. If the people administering the treatment or rating the results 
are aware of the purpose or hypothesis of the study, this may have an 
effect on the results. You might be studying how pairs of people inter- 
act. As part of your study, you might have observers watching the 
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pairs and rating how well each person is communicating. If the ob- 
servers know which participant you expect to get better ratings, the 
observers might unconsciously try to give that participant good rat- 
ings. 

Similarly, if the observers are aware of the hypotheses, they may, 
even without intending to, influence the responses of the participants. 
In studies comparing different language teaching methods, the re- 
searchers themselves often teach the classes that are intended to show 
that one method is more effective than another, and they may influ- 
ence the results. As with participant expectancy, these problems can 
be dealt with by having the people who code the results or administer 
the treatments blind to (that is, unaware of) the hypotheses and to the 
group assignments. 



Measurement Issues 

Another type of threat to internal validity is measurement issues. 
This includes issues related to the instruments used to measure the 
constructs in the study (tests, questionnaires, rating systems, lists of 
questions for interviews, etc.) and the ways they are administered. 

Validity and Reliability 

In the area of measurement issues, there are two components where 
there may be threats to the validity of the results of the research. The 
validity and reliability of the measures themselves may affect the va- 
lidity of the conclusions drawn. In addition, procedures for adminis- 
tering the measures may affect the results. These can all influence the 
validity of a study. 

Validity. Measurement validity is the extent to which the instru- 
ment measures what it is supposed to measure. There are three types 
of measurement validity. Face validity is the extent to which a mea- 
sure “looks” like it measures what it is supposed to measure. This is 
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the logical connection between the construct and the instrument that 
is intended to measure it. A measurement instrument’s having face 
validity means that if you show someone the measurement instru- 
ment, they will understand what it is trying to measure. 

The second type of measurement validity is content validity, which 
is the extent to which an instrument measures all the facets of the 
construct. This is a conceptual problem, not a statistical problem, and 
it depends to a great extent on how the construct is defined. If a mea- 
sure that is supposed to measure communicative ability in a second 
language only tests listening ability, it does not measure all facets of 
communicative ability. 

The third type of measurement validity is construct validity. It is 
related to correlations between the variable in question and other vari- 
ables. There are two types of construct validity. The first is conver- 
gent validity, which means that the variable is correlated with vari- 
ables that it should be correlated with. The second is discriminant 
validity, which means that the variable is not correlated with con- 
structs that it should not be correlated with. For example, measures of 
vocabulary knowledge and reading proficiency might be expected to 
be correlated (convergent validity), but measures of reading profi- 
ciency and communication anxiety would not be correlated (discrimi- 
nant validity). 

Reliability. Reliability, the second aspect of measurement, has two 
aspects. One is stability, that is, consistency over time. For a construct 
that has not changed, there should be a high correlation between a 
participant’s scores at two points in time. One problem related to the 
assessment of the stability of a measure is separating true change from 
lack of reliability. Stability is also evaluated by comparing the results 
of alternate forms of a measure. 

A second aspect of reliability is equivalence, which is the extent to 
which all of the questions in the questionnaire reflect the same con- 
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Struct. For example, if a researcher is measuring attitude toward the 
target language, responses to all of the questions should reflect a similar 
attitude. If the answer to one question seems to reflect a different atti- 
tude, it may be measuring some other construct. Cronbach’s alpha is a 
statistic used to measure equivalence, and if the alpha values reported 
for the questions in a measure are low, the researcher should consider 
removing or revising one or more questions. 

The Practice Effect 

The practice effect is another measurement issue. If more than one 
measurement instrument is used in a study, or if the same instrument 
is used more than once, the results of the instruments administered 
later may be affected. For example, if the same test is administered at 
the beginning and end of a study, having taken the test once already 
may help the participants to do better on it a second time. Using equiva- 
lent rather than identical measures helps to some degree. That is, rather 
than giving the same test twice, you can give two similar tests, one at 
the beginning of the study and a different one at the end of the study. 
However, this does not eliminate the problem entirely. Taking the first 
measure may make participants aware of issues related to the con- 
cept. A control group can be used to identify the practice effect. The 
control group takes the pretest and posttest but does not receive the 
treatment that the other group does. If there is a difference in the re- 
sults of the pretest and posttest for the control group, this might indi- 
cate a practice effect. 

In addition, if you use more than one questionnaire at the same 
time, the results of the first questionnaire might influence the second 
questionnaire. It is therefore a good idea to counterbalance the mea- 
sures, which means that you administer one questionnaire to half the 
participants first and have the other half take the other measure first. 
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Conclusion 

In designing your research, it is vital to limit the factors that might 
influence your results. Among the threats to validity you must con- 
sider are history and maturation effects, the Hawthorne effect, par- 
ticipant and coder or researcher expectancy, and measurement issues 
related to reliability and validity. 

It is impossible to avoid threats to validity entirely. Often in avoid- 
ing one threat to validity, you open up another. For example, in order 
to avoid history effects, you would want to make the study as short as 
possible. However, that might cause a problem with the practice ef- 
fect for your posttest, since the memory of the pretest may still be 
fresh in the subjects’ minds. 

Application Questions 

Consider the design of your research. Which threats to validity 
might be a problem? Is there any way that you can reduce these threats? 

Discussion Questions 

Read the paper in Appendix III. What threats to internal validity 
are there? 
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Sometimes the design of the research can threaten the internal va- 
lidity of a study. The research design might lead to unintentionally 
different treatment of the groups. Some research designs are inher- 
ently faulty because they do not eliminate alternative explanations for 
the results. 

Uncontrolled Differences in the Treatment of Groups 

The first category of design issues is uncontrolled differences in 
the treatment of the groups, that is, situations where the groups in- 
volved in the study are treated in a different way. There are many 
ways in which members of different groups in a research study might 
be treated differently. There are too many to list here, but we will give 
examples of how different participants or groups might be treated dif- 
ferently. 

For example, if you are studying persuasion, and one group hears 
the persuasive message read by one reader, and the other group hears 
it read by a different reader, the groups are being treated differently. If 
you find differences in the effects of the persuasive messages, you 
cannot know whether those differences are due to differences in the 
readers or differences in the messages. This problem should be avoided, 
if possible, by having all the messages read by the same person. 
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Every effort should be made to make the messages identical, ex- 
cept for the characteristic of the message that the researcher is study- 
ing. The messages should be about the same length, and they should 
be read at about the same rate of speaking and with the same intona- 
tion and emphasis. In fact, it might be best to have the message re- 
corded so that each group hears the message read in exactly the same 
way. If one message is longer, or if it is read with greater emphasis on 
certain words, then this might influence the persuasiveness of the 
message. 

Another type of problem in which groups are treated differently 
would be in a study of language teaching methods in which partici- 
pants in the study meet outside of regular classes for instruction using 
a different method. If they are compared with a class that does not 
have any extra instruction, they are receiving more instruction time. 
Both classes have regular classroom instruction, and one class has an 
extra class taught using the method being studied. Therefore there 
must be questions about the validity of the results. In addition to self- 
selection problems (if you are comparing volunteers and non- volun- 
teers), participants who meet outside of regular class time are receiv- 
ing more hours of instruction than those who are only in regular classes. 
If you find that the group using the new method improves more in 
their language proficiency, the difference in the number of hours of 
instruction alone may account for any differences in the improvement 
of their language proficiency. 

If you are using interviews to gather data, you should do the inter- 
views under similar circumstances every time. You should not take 
some interviewees to a quiet room for the interview, and do other 
interviews in a noisy cafeteria. These different settings might result in 
different outcomes for the interviews. 
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Faulty Research Design 

Some research designs are faulty, that is, the research cannot draw 
conclusions from them about the sources of the variance. 

One-group Posttest Only Design 

One faulty design is called a one-group posttest only design. In 
this design, only one group is used, and there is a manipulation or 
series of manipulations followed by a posttest. For example, if a group 
listens to a persuasive message and then their attitude is measured, 
there is no way of knowing whether their attitudes were influenced by 
the message, because nothing was known about their attitudes before 
the message. Based on the results of such a study, no claims can be 
made about the effect of the manipulation. Since nothing is known 
about the levels of the variables before the manipulation, it cannot 
even be claimed that they changed, much less that they changed in a 
particular direction or as a result of the manipulation. 

One-group Pretest/Posttest Design 

A second faulty design is the one-group pretest/posttest design. In 
this design, again, only one group is used, and there is a pretest, a 
manipulation, and a posttest. In this case, the results of the pretest and 
the posttest can be compared, so it can be determined whether there 
were changes in the values of the variables. However, it is not pos- 
sible to draw conclusions about the source of the changes. It may 
have been the treatment. It is also possible that there were history 
effects, maturation effects, or practice effects that explain any differ- 
ences that were found. Again, in a study of persuasive messages, if a 
group’s attitude is measured, and they hear a persuasive message, and 
then they are tested again, no conclusions can be drawn. While their 
attitudes in the second test might be different, there may be other ex- 
planations for the change. 
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Two-group, No Control Design 

The third type of faulty design is a two-group, no control design. In 
this type of study, there is an experimental group (which experiences 
the manipulation) and a nonequivalent control group, with a posttest. 
For example, if one group that hears the persuasive message is a group 
of college students, and the other group is a group of businesspeople, 
the differences might be due to the message, but it might also be due 
to the differences in the people in the groups. Therefore, you cannot 
draw the conclusion that the different persuasive messages had differ- 
ent effects. 



Conclusion 

It is important to design a study so that the groups you are studying 
are treated as similarly as possible. In addition, it is important to de- 
sign studies in a way that you can eliminate other explanations for 
differences between the groups. While it may not be possible to en- 
tirely eliminate all differences between the groups or the way that 
they are treated, you should consider these problems and minimize 
them in designing your study. 

Application Questions 

Consider the threats to validity discussed in this chapter. Are they 
a problem for the study you have designed? Can you change the de- 
sign of your study to eliminate these problems? 

Discussion Questions 

Read the paper in Appendix III. What problems do you see in re- 
search design? 
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Chapter 14 



Analyzing Results of Quantitative Research — 
Descriptive Statistics and Statistical Significance 



Doing quantitative research involves gathering data that can be 
converted into numbers and analyzed. Researchers use statistics to 
analyze the numbers in order to describe the numbers that have been 
collected and find out what the relations are between the numbers. 
Statistics is a complicated subject, and we cannot cover it thoroughly 
in this book. However, in this chapter and the next one, we will intro- 
duce a few basic statistical procedures and statistical terms that you 
might find useful. If you are doing more quantitative research, you 
should learn more about statistics in the future. 



In order to better understand what statistical calculations mean and 
how they work, you might sometimes find it useful to do at least some 
of the simple calculations by hand. Some of the simpler statistical 
formulas will be provided here. However, you will probably usually 
use statistics programs to calculate the results of most of your re- 
search. 

Your school’s computer center may have statistical programs avail- 
able on computers for you to use. SPSS (Statistical Package for the 
Social Sciences) is one of the well-known programs. If your school 
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does not have such programs, you can use statistical programs that 
are available on web pages. One web page that has all the statistical 
calculations that are discussed in this chapter can be found at http:// 
glass.ed.asu.edu/stats/analysis/basic.html. 

Descriptive Statistics 

Descriptive statistics are used to describe what a characteristic is 
like — how high or low the characteristic is, how strong or weak it is, 
what the highest and lowest values are, etc. 

Range 

The range is the distance from the lowest score to the highest score. 
You calculate the range by subtracting the lowest score from the high- 
est score. If the highest score is 15, and the lowest score is 5, then the 
range is 10. 

Average 

There are three types of average, but one that is usually used in 
statistics is called the mean, which is what you probably think of when 
you think of an average. The mean (x, which is pronounced “x bar”) 
is calculated by adding up the scores and dividing the total by the 
number of scores. For example, imagine that your study involves 
measuring the heights of three people. One of the people is 160 centi- 
meters tall, one is 170 centimeters tall, and one is 180. To get their 
average height, you add the three numbers, and then divide the total 
(510 centimeters) by the number of people (3), and you find that the 
mean is 170 centimeters. 

Standard Deviation 

The standard deviation is a statistic that is used to describe how 
much a characteristic differs within the group, that is, whether all the 
members of the group are fairly similar, or whether they are very dif- 
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ferent from one another. If the standard deviation is low, then most of 
the scores are near the mean, and the members of the group do not 
differ much; if it is high, then scores are spread out from the mean, 
and the members of the group are more different from each other. 

Imagine that you are doing a study that includes asking partici- 
pants to rate their ability in English on a scale from 0 to 10. Imagine 
that the mean is 5, and the standard deviation is 1. In that case, about 
68% of the ratings would be within 1 point (one standard deviation) 
above or below the mean, that is, about 68% would be between 4 
points and 6 points. In addition, about 95% of the scores would be 
within two points (two standard deviations) of the mean, that is, about 
95% of the scores would between 3 and 7 points. If the standard de- 
viation is 1 point on a measurement that has a range of 10 points, then 
the responses are fairly close to each other, that is, the participants’ 
ratings of their ability in English are fairly similar to one another. If, 
on the other hand, the standard deviation is 2, then 68% of the partici- 
pants rated their English between 3 and 7 (two points above or below 
the mean, or one standard deviation), and 95% between 1 and 9 (four 
points above or below the mean, or two standard deviations). The 
scores are more spread out, and the participants’ scores are not as 
similar to each other. 



Statistical Significance 

You can also use statistics that compare different values. First, 
however, you have to understand statistical significance. 

When you are comparing two values, you need to find out whether 
they are really different or not. For example, you have asked partici- 
pants to rate their English ability on a scale of 0 to 10. The ratings of 
one group of participants have a mean of 5, and the mean of the other 
group is 6. Is this difference just the result of chance, or is there a real 
difference between these two groups? In technical terms, the question 
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is whether the difference in means is significant. Whether the differ- 
ence is real depends on: 

1) the size of the difference (that is, a difference of 5 points is more 
likely to be significant than a difference of one point) 

2) the number of people in the groups (the larger the group, the more 
likely a difference is significant) 

3) the size of the standard deviation (if there is a small standard devi- 
ation, it is more likely that the difference is significant) 

Significance is reported in terms of probability (p), that is, the like- 
lihood that two groups or two scores are different as a result of chance. 
The value of p is between 0.(X) and 1 .00. Generally, in social science, 
a p of less than .05 is considered significant, which means that the 
difference can be considered real rather than a result of chance. If p is 
equal to or less than .05, it means that in 5 or fewer out of 100 cases, 
the results would have been different — but in 95 out of 100 cases, 
this difference would have been found between the two groups. If p is 
less than .01, then in less than one out of 100 cases, the difference 
would have been found. The lower p is, the more confidence you can 
have that that difference is real and did not happen by chance. 

If the statistical significance is greater than .05, then there you 
cannot say that there is a real difference between the two groups or 
values. 



Conclusion 

Once you have gathered information for your quantitative study, 
you need to use statistics to analyze your results. You use descriptive 
statistics such as the mean and the range to describe the characteris- 
tics of the sample. When you compare two sets of numbers, you also 
need to find out if the difference you see is significant. 
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Application Questions 

Imagine that you have shown two groups of people a picture of a 
woman. One group (suit group) saw a picture of the woman wearing 
a suit. The other group (t-shirt group) saw a picture of the woman 
wearing a t-shirt. You asked the groups, among other questions, to 
rate how friendly the woman was. (In your scale, 1 = very unfriendly 
and 5 = very friendly.) There are ten people in each group, and you 
get the following results: 

T-shirt group 

4533454345 

Suit group 

3123443432 

What is the range of each group? What is the mean? What is the 
standard deviation? What are the differences between the groups? (The 
answers can be found in Appendix VI.) 

Which of these statistical procedures would you use for your study? 



Chapter 15 

Analyzing Results of Quantitative Research — 
Comparative Statistics 



Keywords 
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Generally you design a quantitative study so that you can compare 
two or more sets of numbers. The following statistics are ones that 
you can use to compare different things. Using comparative (or infer- 
ential) statistics, you can compare the scores of two different groups 
on the same measure. For example, you can compare the English pro- 
ficiency of two different groups. Also, you can compare two different 
scores for the same group of people. You can compare a group’s scores 
on English proficiency and their grades at an American university. In 
this chapter, you will learn about some comparative statistics. 

Comparative Statistics 

TTest 

In some cases, your hypothesis states that one group will be higher 
or lower than the other on a certain variable. For example, your hy- 
pothesis might be that participants who hear Message A will find it 
more persuasive than those who hear Message B. The t test is used to 
compare two means, that is, to see whether the means are signifi- 
cantly different. If you are comparing two groups, you can use a t test. 

For example, one group hears a persuasive message that appeals 
to emotion, and the other group hears a persuasive message that ap- 
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peals to logic. You measure the persuasiveness of the message using a 
5-point scale. You find that the former group had a mean of 4.5, and 
the latter group had a mean of 3.5. You use a t test to find out whether 
the difference is significant. T will be a positive number greater than 
one. If you use a computer program to calculate it, the printout you 
get will tell you what the probability (p) is, so that you can tell whether 
it is significant. As discussed in the previous chapter, if p is less than 
.05, then the difference is significant. That means that the effect of the 
persuasive message on the two groups was different. 

You can calculate the t test at http://nimitz.mcs.kent.edu/~blewis/ 
stat/tTest.html. 

Chi-Square 

Chi-square ( 0 is a statistic that is used when you want to com- 

pare the number of items in different categories. For example, you 
might be doing a study comparing three advertisements. You ask 30 
participants to decide which is the most persuasive advertisement. 
The following chart indicates how many participants thought each 
advertisement was most persuasive. 

Advertisement A — 4 

Advertisement B — 19 

Advertisement C — 7 

As with the t test, you would calculate chi-square and the statistics 
program would tell you whether the difference among the ratings was 
significant. If the chi-square tells you that there is a significant differ- 
ence, then you will know that Advertisement B was the most persuasive. 

You can also use chi-square when you divide the participants into 
groups. For example, if you were interested in whether there were 
differences among men and women in the way they rated the adver- 
tisements, you could also use chi-square. In this case, the number of 
male and female participants who liked each advertisement best might 
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look like this: 

Female Male 

Advertisement A — 1 3 

Advertisement B — 12 7 

Advertisement C — 2 5 



Again, you would have calculate chi-square for these numbers to 
see if there is a difference among the ratings. 

You can calculate the chi square at http://www.georgetown.edu/ 
cball/webtools/web_chi.html. 

Pearson Correlation 

In some cases, your hypothesis states that the independent vari- 
able and the dependent variable will go up or down in relation to each 
other. That is, if the independent variable goes up, the dependent vari- 
able goes up, and if the independent variable goes down, the depen- 
dent variable goes down (a positive correlation). It can also mean that 
if the independent variable goes up, the dependent variable goes down 
(a negative correlation). For example, you might have a hypothesis 
that, among Japanese college students in the US, those who rate their 
English proficiency as being higher will have more American friends, 
and those who rate their English proficiency as being lower will have 
fewer friends. 

In order to test that hypothesis, you might ask participants to rate 
their English proficiency and ask them how many American friends 
they have. Then you would calculate a Pearson correlation (r) for the 
two sets of numbers. A Pearson correlation would tell you whether it 
was true that students who think they speak English better have more 
friends. The value of r ranges from -1.00 to 1.00. If r is between .01 
and 1 .00, then people who rate their English higher have more friends, 
and the variables are said to be positively correlated. If p is .05 or less, 
then the difference you found is significant. 




T13 
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If, on the other hand, you find that the correlation between the two 
values is between -1.0 and -.01, this is called a negative correlation. It 
means that as the independent variable goes up, the dependent vari- 
able goes down and vice versa. That is (if p is .05 or less), this means 
that people who rate their English as being lower have more friends. 
(If the correlation is 0.00, there is no relationship at all between the 
variables, that is, the relationship is completely random.) 

You have to keep in mind that even if two values are correlated, it 
does not mean that one is the cause of the other. Based on a positive 
correlation in this study, you cannot say that better English proficiency 
causes students to have more American friends. It is possible that they 
have greater confidence in their English proficiency as a result of 
making American friends and interacting with them. 

You can calculate Pearson correlations at http://faculty.vassar.edu/ 
lo wry/corr_stats .html. 

Partial correlation. In some cases, two variables will appear to be 
correlated with each other only because they are correlated with a 
third variable. For example, if you are studying Japanese students in 
American universities, you might find that their English proficiency 
and their grades are correlated, that is, students with good English 
proficiency also get good grades. However, it might be that both Eng- 
lish proficiency and grades are correlated with intelligence, in which 
case, they are not really correlated with each other. If you believe this 
might be the case, you will calculate a partial correlation. 

To calculate a partial correlation, you would measure the English 
proficiency, the grades, and the intelligence of the Japanese students. 
These values are used to calculate a partial correlation. Partial corre- 
lations are interpreted in the same ways as the Pearson correlation, 
that is, they are between -1.00 and 1.00. You may find, for example, 
that the correlation between English ability and grades is .78, between 
intelligence and grades is .83, and between English ability and intelli- 
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gence is .87, all very strong positive correlations. If you calculate the 
partial correlation for English ability and grades, you can find out if 
there is a real correlation between English ability and grades, or 
whether they are unrelated to each other, but both related to intelli- 
gence. If you find that the partial correlation for English ability and 
grades is .09 (and p is greater than .05, that is, the correlation is not 
significant), this indicates that there is no real correlation between 
English ability and grades. They only appear to be correlated, be- 
cause they are both correlated with intelligence. However, if the par- 
tial correlation is .73 (and p is less than .05), this means that there is 
still a real correlation between English ability and grades, indepen- 
dent of both their correlations to intelligence. 

You can calculate partial correlations at http://faculty.vassar.edu/ 
lowry/par.html. 

ANOVA 

Sometimes you want to compare two variables for two groups. To 
do this, you use an analysis of variance (ANOVA). For example, if 
you are comparing the effectiveness of logical and emotional persua- 
sive messages, you might state in your hypotheses that women will 
rate logical messages as being more persuasive and men will find emo- 
tional messages more persuasive. 

Logical message Emotional message 

Male 

Female 

In this situation, you have two variables (sex [male or female] and 
message [logical or emotional]). You also have four conditions: 

A. males who listen to logical messages 

B. males who listen to emotional messages 

C. females who listen to logical messages 



A 


B 


C 


D 




115 



100 Part II Quantitative Research 



D. females who listen to emotional messages 

ANOVA uses the mean for each variable and for each condition 
(males/logical messages, males/emotional messages, females/logical 
messages, females/emotional messages) and determines whether each 
variable has an effect. For example, if you did the calculations and 
found that the means for sex were significantly different (that is, if p 
was less than .05 for sex), that tells you that males and females were 
different in how they rated the persuasiveness of the messages. (You 
look at the means for males and females to see whether males or fe- 
males rated the messages as being more persuasive.) If you find a 
significant difference in the ratings for message, then the messages 
were rated differently. (Simil 2 irly, you look at the means for the logi- 
cal and emotional messages to see which was rated higher.) 

In addition, you may find that the emotional message was rated 
higher, but only by males. This is called an interaction effect. This 
means that males found the emotional message more effective than 
females did, but there was no difference in their ratings of the logical 
message. Two things (being male and an emotional message) were 
necessary for the effect. 

You can calculate ANOVA at http://faculty.vassar.edu/lowry/ 
VassarStats.htm. 



Reporting Scores 

After you have calculated the statistics, you need to report them in 
a way that is clear to anyone who is reading about them. One way to 
report them is to use tables. For example, if you are reporting means 
and standard deviations for different groups or for different variables, 
a table might look like this. 



Table 1. 

Means and standard deviations 
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Variable 


Mean 


S.D. 


A 


5.23 


1.12 


B 


10.21 


3.31 


C 


4.43 


0.34 



When you are reporting Pearson correlations, you usually make a 
grid like this; 

Table 2. 

Pearson correlations 





A 


B 


C 


A 


1.00 


.76 


.02 






II 

b 

o 


(P = .36) 


B 


.76 


1.00 


-.09 




o 

q 

II 




(P = .22) 


C 


.02 


-.09 


1.00 




(P = .36) 


(P = .22) 





If you look at the intersection between A and B, you will see that 
the correlation between these two variables is .76, and the probability 
is .001, meaning that the correlation is significant. 

Another way to report your data is to use graphs. There are several 
types of graphs, including the bar graph and the line graph. They use 
bars or lines to show the relative size of the variables. A bar graph can 
be used, for example, to show how two groups compared on different 
variables. If you measured the English language proficiency, grades, 
and intelligence of two groups, you could show how they compare on 
a bar graph. A line graph is usually used to show changes over time. 

.117 
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For example, if you want to show how the groups’ average score on 
the TOEFL changed over time, you might use a line graph. A pie chart 
can be used to show what percentage of participants fall into different 
categories. You can use Microsoft Excel to make graphs and pie charts. 

The following are some examples of pie charts and bar graphs. 
The first graph, a bar graph shows the levels of six variables for two 
groups. The second graph shows the changes in two variables at six 
points in time on a line graph. The pie chart shows what percentage of 
the participants fall into each category. 
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Conclusion 

In order to compare different groups or variables, you use such 
statistical tests as the t test, chi-square, a Pearson correlation, and 
ANOVA. When you report the results of the statistical tests, you need 
to use tables and possibly graphs to make their meaning clear to the 
reader. 

Application Questions 

Look again at the data from the application question for the previ- 
ous chapter. Perform a t test to determine whether the difference be- 
tween the means is significant. What does that result mean? 

In the example in the reading, is there a significant difference in 
the effectiveness of Advertisements A, B, and C? Calculate chi-square 
to find out. What do the results mean? 

You are doing a study of the relationship between listening skill 
and reading skill in a second language. You give 10 participants a 
listening test and a reading test. The following are the scores. Calcu- 
late the Pearson correlation to find out what the relationship is. 
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Participant 


Listening 


Reading 


1 


52 


71 


2 


59 


69 


3 


72 


80 


4 


77 


91 


5 


74 


95 


6 


63 


79 


7 


44 


58 


8 


54 


56 


9 


31 


45 


10 


51 


61 



Participants 1-5 are from Germany, and participants 6-10 are Japa- 
nese. Perform an ANOVA to find out whether there are differences 
between the Japanese and German participants, whether there is a 
significant difference between reading and listening proficiency (as- 
suming that the tests are equally difficult), and whether there is an 
interaction between the two. 

(The answers can be found in Appendix VI.) 

Which of these statistical procedures would you use for your study? 

Discussion Questions 

Read the paper in Appendix III. What statistics are used, and what 
do the results mean? 



Part III 

Qualitative Research 
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Designing Qualitative Research Studies 
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When you do quantitative research, control of the circumstances 
of gathering data is very important, and data is gathered in a way that 
allows the researcher to convert the information into numbers and 
statistically analyze them. When you do qualitative research, there 
are different priorities. It is more important to look in depth at what 
people do, how they communicate, etc. In this chapter, we will dis- 
cuss how qualitative studies are designed. 

Steps in Qualitative Research 

Qualitative research methods are not as well established as quanti- 
tative research methods. However, there are some steps that qualita- 
tive researchers go through. 

1 . Decide on an area of interest. In quantitative research, you often 
start with a hypothesis or, at least, research questions. In qualitative 
research, hypotheses are rarely used. Researchers often just have an 
area of interest that they would like to pursue, such as how members 
of the tennis club at a Japanese university interact or how an Ameri- 
can university admissions committee makes decisions. However, you 
might find it helpful in focusing your research to have research ques- 
tions, such as “What factors are involved in admissions decisions and 
which ones are most important?” 

2. Decide how to gather information and choose participants. Your 
next step is to decide how to gather information. As with quantitative 
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research, you can gather information through observation or through 
interviews or questionnaires. Observation and interviews are prob- 
ably more conunon for qualitative research than questionnaires. In- 
terviews are preferred to questionnaires, because they allow the re- 
searcher to ask for more information about interesting responses. 

The procedures for doing interviews or questionnaires are similar 
to those described previously for quantitative research. (See chapters 
8 and 9 for a discussion of using interviews and questionnaires.) How- 
ever, the questions are generally open-ended, and it is not necessary 
for the responses to be able to be converted into numbers. Instead, the 
emphasis should be on forming open-ended questions that will elicit 
honest, extensive, and relevant responses. 

Observation is a particularly important method of gathering infor- 
mation for qualitative research. There are important differences in 
how observation is done in quantitative and qualitative research. Pro- 
cedures and related issues are discussed in chapters 17 and 18. 

Quantitative researchers try to choose participants at random, so 
that they can study “typical” members of the population they are in- 
terested in. Qualitative researchers have a variety of ways to choose 
their participants. (This is not an exhaustive list of the possibilities; it 
is only some possibilities of how to choose participants.) 

a. Maximum variation. Instead of studying “typical” members of 
the population, researchers might choose to study the most ex- 
treme members of the population and see how they are different 
from each other. For example, a researcher might study univer- 
sity students who were most involved in activities on campus 
(clubs, student government, volunteering, etc.) and the students 
who were not involved in any activities. 

b. Networking. Participants are found by asking each participant 
to refer the researcher to other possible participants. This is some- 
times the only way to find participants if you are studying a 
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group with rare characteristics, but who might know each other. 
For example, parents who teach their children at home gener- 
ally know other parents who teach their children at home. If you 
are studying such families, the best way to find them might be to 
ask the families you are already studying. This approach is also 
sometimes used when researchers are interested in understand- 
ing how networks of people with similar interests fit together. 

c. Extreme cases. Some researchers study only the most extreme 
cases, such as only university students who are the most involved 
in campus activities, or only students who participate in no cam- 
pus activities at all. 

d. Typical cases. Some researchers decide what is average or typi- 
cal and then find some participants to fit those criteria. For ex- 
ample, a researcher might use a questionnaire to find out how 
many hours a week, on average, university students spend on 
campus activities outside the classroom and find students who 
fit that average. 

e. Unique case. Sometimes a researcher will choose one case that 
is very unusual, perhaps one that he/she finds entirely by acci- 
dent. For instance, the researcher might find a blind student who 
is active in clubs, student government, and doing volunteer work. 

f. Ideal case. Another possibility is to use someone who is in a 
perfect situation, such as the interaction in the classroom of an 
award-winning teacher. 

3. Observe/ask open-ended questions (interview or questionnaire). 
The next step is to carry out the observation or do the interviews or 
questionnaires. 

4. Record observations/interview. If you are doing an observation 
or interviews, you need to record and/or transcribe your results. If 
you are not observing as a participant, you can take notes as you ob- 
serve. However, you might want to read over your notes as soon as 
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possible after the observation to see whether you remember anything 
you did not have a chance to write down. If you are observing as a 
participant, you will probably not have an opportunity to take notes at 
the time you are observing, but you should do so as soon as possible 
afterwards. 

If you are using interviews in your research, it is probably better to 
videotape or audiotape the interview. Videotaping may be technically 
more difficult than audiotaping, and being videotaped may make 
interviewees self-conscious. However, if you videotape the interview, 
you can look at such nonverbal aspects of the interview as facial ex- 
pressions and gestures. In either case, you need to transcribe the ques- 
tions and answers from your interviews. 

5. Begin to analyze data. You can begin your analysis even before 
you finish gathering your data. This will allow you to know whether 
you are getting the type of information that you want. Analysis of the 
data in qualitative research is discussed in detail in chapters 20 and 
21 . 

6. Return to observe or ask more questions. You may find soihe- 
thing you want to learn more about, or you may not be getting the 
kind of information you were hoping for. You can go back to the 
interviewees again to ask more questions, or you can change the way 
you are doing the observation to get more information. 

7. Repeat the cycle of observation/questions. Continue the process 
until you have as much information as you need. 

8. Summarize the findings and draw conclusions. Based on the 
results of your analysis, what conclusions can you draw? What did 
you learn about the area you were interested in? 

Conclusion 

The procedures for qualitative research involve choosing an area 
of interest and doing observations, interviews or questionnaires to 
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learn about that area in an in-depth way from the point of view of 
those who are involved. Unlike quantitative research, it does not nec- 
essarily involve studying what is “typical.” It may involve interview- 
ing or observing more than once in order to gain a greater depth of 
understanding. 

Application Questions 

What topic do you think you would like to study, using a qualita- 
tive approach? What would be the most appropriate way to study it? 
(You might want to use more than one method.) 

Write some questions for an interview and a questionnaire to study 
the topic you are interested in. 

Discussion Questions 

Read the paper in Appendix III. In what way does the researcher 
use qualitative approaches? 



o 
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Observation is used in both quantitative and qualitative research. 
However, it is not used in the same way. In this chapter, we will dis- 
cuss how observation is used differently in these two approaches and 
describe the use of observation in qualitative research. 

Observation in Qualitative and Quantitative Research 
Quantitative Research 

As was discussed in Chapter 10, researchers using a quantitative 
approach use observations as a means to count behaviors. In addition, 
in quantitative research, the emphasis is on control. (For a discussion 
of control as it applies to quantitative research, see Chapter 11.) The 
researcher attempts to observe different people under as nearly as pos- 
sible the same conditions. 

In addition, the quantitative researcher generally observes from 
the “outside.” The researcher tries to avoid influencing the situation 
being observed. Even if a confederate (a person who is helping the 
researcher) is used to create a particular situation, the confederate only 
acts in ways that are decided by the researcher. The purpose of using 
a confederate is to maintain better control. For example, if you were a 
quantitative researcher studying refusals, you might have a confeder- 
ate make requests and observe the resulting refusals. However, you 
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would tell the confederate exactly what to say and exactly how to 
respond to the refusal. The confederate makes an effort to use the 
same words and the same vocal and facial expression every time he/ 
she makes the request. 

Finally, quantitative research needs a large number of observations, 
so emphasis is on making a large number of relatively brief observa- 
tions. 

Qualitative Research 

There are many situations in which you might find observation 
useful as a researcher taking a qualitative approach. If you want to 
know about communication among club members at a university, you 
might use observation. If you want to study how decisions are made 
by a university admissions committee, you might use observation. If 
you want to see how children and their parents communicate about 
rules of behavior, you might use observation. 

In contrast to a quantitative approach, a researcher using a qualita- 
tive approach does not attempt to exercise control in the sense of 
making observations under the same conditions every time. A quali- 
tative researcher is more interested in observing a natural situation 
than in exercising control. 

In addition, qualitative researchers believe that in many situations, 
in order to understand the dynamics of a group, a researcher must be 
part of that group. Therefore, a qualitative researcher may participate 
in the group to some extent, and may even become an active member 
of the group. For example, a researcher might want to try to under- 
stand the communication between supervisors and factory workers, 
and may therefore get a job as a worker in a factory in order to ob- 
serve that communication from the point of view of a factory worker. 
As a participant, the researcher may influence the communication as 
well as observe it. 

In contrast to quantitative research, a qualitative researcher may 
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only observe one situation, but will observe it in great depth. Observ- 
ing and comparing a large number of situations is not necessary. In- 
stead, the researcher spends a lot of time observing just one or two 
situations. 

Advantages and Disadvantages of Observation 

Advantages 

Observing a group can give you greater insight into the interaction 
of that group than just asking members about how they interact. Of- 
ten members do not have much awareness of the ways in which they 
interact. If you can actually observe them, you will learn much more. 

Observation can also allow you to understand group members from 
their own point of view, particularly if you observe as a participant in 
the group. 

Observation is particularly useful in understanding the non-verbal 
aspects of interaction, such as gestures and facial expression. These 
are aspects of communication that the participants themselves may 
not be aware of, so the only way to understand them is to observe 
them. 

Finally, you can see how the group behaves in its natural setting. 
This will give you the most accurate impression of how groups interact. 

Disadvantages 

The main disadvantage of observation is that the presence of the 
observer may distort the interaction among the group members. The 
fact that there is an observer there may make people act differently 
than they otherwise would. This is especially true if the observer is a 
complete outsider, for example, if a Western anthropologist is study- 
ing a tribe in South America. It is less of a problem if a university 
student is studying interactions in a university club. 

As in any research, the researcher brings his/her own preconcep- 
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lions and ideas to the research. These may affect what observers see 
and how they interpret what they see. 

Coding the results of observational research can be very difficult. 
Observation, especially over a long period of time, produces a lot of data. 
Finding trends in the data and making sense of it can be difficult. 

Finally, small sample sizes may be a problem in observational re- 
search. It is difficult to do in-depth observations on a large number of 
people, so observations are done on a small number of people. This 
raises the question of whether the people being observed are typical. 
One way of dealing with this disadvantage is to combine it with other 
methods. For example, you can do interviews with other people to 
find out how typical your observations were. 

Conclusion 

For qualitative researchers, observation is an important tool. In 
contrast to observation used for quantitative research, it is less con- 
trolled. While it has advantages, it also has difficulties, and in choos- 
ing to do observation for qualitative research, you need to be aware of 
both. 

Application Questions 

What kinds of observation might be useful for the qualitative re- 
search you are planning to do? 
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Observational research is a useful and important tool in qualitative 
research. However, it can involve many practical problems. In this 
chapter, we will discuss what some of those problems are and how to 
solve them. 

Types of Observation in Qualitative Research 

When you are planning to do observational research, you need to 
make some decisions about the degree to which you will participate. 
You also need to decide whether the people you are observing will 
know whether you are observing them. 

Observation in qualitative research can be divided into three types: 
participant observation, non-participant observation, and ethnogra- 
phy. Obviously, participant observation involves having the observer 
actually become a member of the group that is being observed. Non- 
participant observation, on the other hand, is a situation where the 
researcher observes a group as an outsider, without participating in 
any group activities. Ethnography is a form of research where the 
researcher investigates social interaction among members of little- 
known social groups. 
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Participant Observation 

Participant observation is often used to study groups where not 
much is known about a group, and where group members may be 
reluctant to talk about the group. It is used to compile data related to 
nonverbal behavior, such as gestures or facial expression, as well as 
such topics as how groups of people interact or how communication 
is used among supervisors and their employees. 

Participant observation is a very useful technique, because it al- 
lows researchers to see the group being studied from the point of view 
of an insider. This allows the researcher to gain insights and under- 
standings that no outsider could have. However, one danger of par- 
ticipant observation is that the researcher might identify too closely 
with the group. In that case, the researcher can only see the group’s 
viewpoint, not an outsider’s viewpoint. 

Covert and overt observation. Participant observation can be fur- 
ther divided into two types — covert and overt observation. In the 
case of covert observation, those being observed do not know that a 
person participating in the group is also a researcher. In the case of 
overt observation, the members of the group are aware of the re- 
searcher’s dual role of participant and observer. 

Both approaches have disadvantages. If you are using covert ob- 
servation, there are ethical problems, that is, it might not be ethically 
acceptable to observe people without their knowledge. On the other 
hand, with overt observation, people might behave differently know- 
ing that they are being observed. However, participants may become 
accustomed to being observed over time. 

Influences on participant observation. In doing participant ob- 
servation, the researcher should be aware of outside influences that 
might have an effect on what is being observed. One such influence is 
the season or time of year. For example, if the researcher observes an 
accounting firm just before the deadline for turning in tax forms, the 
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observations are likely to be different than they would be at a different 
time of year. 

The researcher her/himself may also have an influence on the ob- 
servations. As mentioned above, in the case of overt observation, some 
people might behave differently knowing that they are being observed. 
In the case of covert observation, the researcher’s behavior as a mem- 
ber of the group might influence the way other members act. 

Non-Participant Observation 

In some cases, researchers observe behavior without taking part in 
it. Like participant observation, this can be done either covertly or 
overtly. It allows the researcher to observe the group interaction in a 
natural setting, but only as an outsider. 

Ethnography 

Ethnography combines participant observation with interviews as 
a researcher lives with and observes a social group. It is traditionally 
used by anthropologists on isolated groups of people, such as Indians 
in the rain forests of South America or tribes on islands in the South 
Pacific. Margaret Mead was an anthropologist who is well known for 
studying and writing about such groups. Ethnography has also some- 
times been used with such groups as gang members in inner cities. 

The emphasis in ethnography is understanding a culture, includ- 
ing what the people do (cultural behavior), what they know (cultural 
knowledge), and what they make and use (cultural artifacts). 

Choosing the Degree of Participation 

When planning your research, you need to decide whether you 
will be a participant. In some cases, participation is not possible. For 
example, if you are observing interaction among students in a kinder- 
garten, you will not be able to participate. However, if you are a col- 
lege student, and you want to observe the interactions of a club, you 
can join the club and be a participant. If participation is possible, you 
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need to decide whether you can observe more effectively if you act as 
a participant or as a non-participant. 

Gaining Access 

Once you have decided what and whom you are going to observe, 
you need to gain access to that group so you can do your observa- 
tions. There are two aspects to this step in your research — entry and 
permission. 

Gaining Entry 

Depending on the type of group you are planning to observe, you 
have to find a way to see them in their natural setting. Some groups 
can be observed in public. For example, if you want to observe fan 
behavior at high school baseball tournaments, you can easily watch 
fans just by going to the stadium. Some groups, however, meet in 
private, and you have to find a way to observe them. If you want to 
study the decision-making in a university club, you can join the club. 
If you want to study interaction between supervisors and workers in a 
fast food restaurant, you can get a job at a fast food restaurant. If you 
want to study interaction between teachers and students in a class- 
room, you can ask permission of the teacher to sit in on the class and 
observe. 

Getting Permission 

If you are observing a group in public, such as baseball fans at a 
baseball game, you do not need to get permission. However, if you 
are doing observation in an overt way of a closed group, you need to 
ask permission. If you want to observe in a classroom, you need to 
ask permission of both the principal of the school and the teacher 
whose class you want to observe. 

If you are going to do covert observation, you might not be able to 
ask permission of anyone you will be observing. However, if you are 
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observing people in the organization, it might be best to ask permis- 
sion of someone higher in the organization. For example, if you are 
observing interaction among employees at a business, you might ask 
permission of the manager. 

When you ask permission to do observation, you should explain 
briefly what kind of study you are doing, what you want to observe, 
and how you will use your observations. You may need to promise 
that the organization will be anonymous and real names will not be 
used. 

In some cases, it may be impossible to get permission to observe 
the group you are interested in. A company might not want to give 
you access to their decision-making process. 

Combining Research Techniques 

It is often useful to combine observation with other research tech- 
niques. Observation can be used either before or after such techniques 
as interviews or questionnaires. On one hand, you can use interviews 
or questionnaires before observations to help you understand what to 
look for when you do your observations. Alternatively, you can use 
questionnaires or observations after the observations to help you un- 
derstand the meanings of your observations. 

For example, if you are studying how a university admissions com- 
mittee makes decisions, you might first give them a questionnaire 
about the basis on which they make their decisions. You might use 
this information to help you when you are analyzing the results of 
your observations. However, you should be careful not to be limited 
by the results of the questionnaire. You might find from your observa- 
tions that members of the committee used factors that they did not 
mention in the questionnaire. 

On the other hand, after you have made your observations, you 
might want to interview the committee members to find out more 
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about why they made the decisions that they did, or ask one commit- 
tee member what he/she meant by a particular comment. 

Conclusion 

There are three ways of gathering data through observation — par- 
ticipant observation, non-participant observation, and ethnography. 
Participant and non-participant observation can be done either overtly 
or covertly. A researcher needs to make decisions about these, as well 
as to decide how to gain entry and to get permission to do the obser- 
vations. 

Application Questions 

What kind of observation would you use for your study, partici- 
pant observation, non-participant observation, or an ethnographic ap- 
proach? Would you observe overtly or covertly? Why? 

How would you ask permission? Who would you ask for permis- 
sion? 

Discussion Questions 

Read the paper in Appendix III. How could the researcher have 
used qualitative approaches to observation? 
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Sometimes a researcher is interested in a particular situation. In 
that case, the researcher may do a case study, that is, combine various 
techniques of gathering data to look at that particular situation in depth. 
In this chapter, we will discuss case studies and how they are done. 

Definition of a Case Study 

The case study approach involves gathering and presenting as much 
information as possible about a particular situation, small group, or 
individual. For example, a qualitative researcher using a case study 
approach might study the students who live in a dormitory to find out 
how they relate to each other. Another type of case study might look 
at a particular situation. One famous case study about a situation was 
done in the 1970s. When someone put poison in a certain type of 
medicine, the company that manufactured the medicine did a good 
job of reassuring the public. A case study was done about how the 
company reacted to the crisis. 

The emphasis of case studies is on describing in detail that one 
individual or group, rather than on making generalizations that apply 
to other groups or contexts. They do not usually look for cause-and- 
effect relationships; rather, the case study approach emphasizes de- 
scribing, exploring, and understanding the person or group under study. 
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Case studies are similar to ethnography in that they involve collecting 
information mainly through interviews, observation, etc., but ethnog- 
raphy usually involves gathering information about a culture, while a 
case study is usually about one or more small groups, situations, or 
individuals. 

Case studies have the advantage of being able to look into the re- 
search questions in great depth. They can provide a great deal of in- 
formation about the context. However, their disadvantage is that they 
involve one particular situation and do not generalize beyond that 
particular situation. Also, they are also criticized as being subjective. 
However, the validity of a study can be maximized by using two or 
more methods of gathering data and getting feedback from the par- 
ticipants about the accuracy of the results. 

Case studies can look at an individual and their language develop- 
ment, learning, and interpersonal interactions. One Japanese linguist 
did a study of how her daughter learned English request forms while 
they were on sabbatical in Australia. This would be an example of a 
case study on an individual. They can also look at organizations, their 
communication networks, structure, etc. For example, a qualitative 
researcher might do a case study about how decisions are made in a 
company. Or they may look at a group and its communication pat- 
terns, decision-making processes, and so on. 

Types of Case Studies 

There are four basic types of case studies. 

Illustrative Case Studies 

An illustrative case study is a basic kind of case study used to 
show what a group or situation is like. 

Exploratory Case Studies 

Exploratory case studies are small studies that are done as a first 
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Step before a larger study. They are used to help the researcher find 
out what kinds of issues and questions she/he wants to look at in more 
detail in the larger study. For example, a researcher looking at Ameri- 
cans living in Japan and the social networks that they form might not 
be certain what sorts of issues will be raised in the study and might 
want to do a brief case study to find out more about what kinds of 
questions to ask or what to observe. 

Cumulative Case Studies 

Cumulative case studies put together information from different 
studies. In cumulative case studies, a researcher reads several studies 
done on similar groups of people, or similar situations. This allows 
greater generalization than just one study will allow. For example, a 
researcher might look for all of the studies on the social networks of 
people who live in foreign countries and summarize them. 

Critical Instance Case Studies 

Critical instance case studies are case studies of situations that are 
unique and not able to be generalized to other situations or groups of 
people. For example, if a researcher is studying the language devel- 
opment of a child whose mother is a native Chinese speaker, whose 
father is a native Japanese speaker, and who lives in Germany and 
attends an English-medium school, this would be a critical instance 
case study, because this is an unusual situation. 

Designing a Case Study 

There are so many different kinds of case studies that it is difficult 
to make general guidelines about them. However, when planning a 
case study, there are typically three things to decide: 

1. Decide what questions to study. As with other qualitative ap- 
proaches, researchers who use case studies usually do not make hy- 
potheses. However, research questions are generally formed to guide 
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the study. 

2. Decide: 

a. how to collect data to answer the questions. There are various 
ways that data is collected for case studies. A researcher can use 
documents or other records, interviews, questionnaires, obser- 
vation (participant or non-participant), protocol analysis (that 
is, analysis of transcripts of participants talking about how they 
do a task as they are doing the task), etc. Qualitative researchers 
generally prefer to use at least two methods to gather data for a 
case study, and sometimes more. Part of the analysis of the data 
can include comparing the information gathered in different 
ways. 

b. what person, group, or organization to collect it from. Case studies 
can use one participant or a limited group of participants, but 
they do generally not use many participants, usually fewer than 
ten. Participants can come from a cross section or they can all 
be similar to each other. Researchers often do a brief “case his- 
tory” of the participants in order to better understand their back- 
ground and how it might affect the outcome of the study. 

3. Decide how to analyze the data. Methods described in Chapters 
20 and 21 can be used to analyze the data for case studies. One differ- 
ence between case studies and other forms of qualitative research is 
that a case study is often told as a narrative, that is, a story in time 
order. 



Writing Up Case Studies 

Whatever form a case study takes, an important part of the writing 
up of a case study is description. Case studies generally use what is 
called “thick description,” that is, very detailed description of a per- 
son or group, the context, the situation, and every possible relevant 
detail. It may also include the participants’ reactions to the results of 
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the study and suggestions for future research. 

Conclusion 

Case studies are a useful approach when you want to describe in 
detail a person, group, or situation. When you do a case study, you 
gather information through interviews, protocol analysis, observation, 
etc. When you analyze and write up a case study, you use “thick de- 
scription” of the context, the participants, etc. 

Application Questions 

How would you use a case study approach to answer your research 
questions? Who would you do your case study on? How would you 
gather your information? 
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Once you have gathered your data for your study you need to ana- 
lyze it. Qualitative research often involves the analysis of texts. Texts 
can be many different things — written answers to a questionnaire a 
transcription of interviews, a transcription of a conversation among 
members of a group, or a researcher’s notes on participant or non- 
participant observations. In this chapter, we will discuss six techniques 
for analyzing texts, and in the next chapter, we will discuss six more. 

The major emphasis in qualitative analysis of data is finding 
“themes,” that is, ideas that occur over and over in the data. There are 
four categories of approaches — word-based techniques, scrutiny- 
based techniques, techniques based on linguistic features, and tactile 
or manipulative techniques. You can use different ways to analyze 
data, depending on what the researcher is interested in. In this chap- 
ter, we will discuss word-based techniques and scrutiny-based tech- 
niques. In the next chapter, we will discuss the latter two. 

Word-Based Techniques 

One approach to data analysis is to look at the words themselves. 
These word-based techniques can identify recurrent themes. They are 
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mainly used as a way to start looking at the data and identify some 
themes that the researcher will then look into in more detail. 

Word Repetitions 

One simple way of analyzing texts (that is, what people have writ- 
ten or said in response to interview or questionnaire questions) is to 
look at words that people use frequently. This technique is based on 
the assumption that people repeat words that express ideas that are 
important to them. Therefore, if you can find the words that people 
repeat frequently, you will be able to find out what is on their minds. 

For example, in one study, researchers asked parents to describe 
their children. They found that mothers frequently used such words 
as “friends,” “creative,” and “honest.” Fathers, on the other hand, fre- 
quently used such words as “school,” “good,” and “student.” Using 
these different sets of words, the researchers identified different themes 
in the ways mothers and fathers talked about their children. 

Indigenous Categories 

Another way to find themes is to look for words that are unfamil- 
iar or for familiar words that are used in an unusual way. This is a 
useful technique when studying texts produced by a particular group 
that may have its own vocabulary or may use common words in un- 
usual ways. These can point to themes that indicate how people in the 
particular group are different from other people. This method of analy- 
sis has been used on studies of medical students, drug addicts, and 
homeless people. In the study of homeless people who were talking 
about their experiences, the researcher found many references to “mak- 
ing a flop.” “Flop” is a noun that the homeless people use for “bed” or 
a verb that they use for “sleep.” The researchers analyzed the various 
ways that “flop” was used (kinds of flops, ways to make flops, people 
who bothered them when they flopped, etc.) to leam about the way 
homeless people talk about how and where they sleep — an impor- 
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tant issue for someone who does not have a home. 

Key-words-in-context (KWIC) 

The key-words-in-context method, the researcher chooses key 
words and then does a systematic search to find all of the examples of 
those key words in the text. (This is done with a computer word search 
of a file.) The words are sorted according to how they are used, whether 
the word is used in a positive or negative way, etc. 

Scrutiny-Based Techniques 

Scrutiny-based techniques involve not just looking at the words 
themselves but at the meanings that re-occur in the texts. The researcher 
can find some themes that were suggested by one or more of the word- 
based techniques, plus additional ones. 

Compare and Contrast Method 

When using the compare and contrast method, the researcher reads 
the texts and looks for similarities and differences between different 
texts or between texts written by different kinds of people. The re- 
searcher might look at transcriptions of interviews with men and 
women who are talking about relationships and try to find ways that 
they talk about the relationships differently. 

Social Science Queries 

In this approach, researchers look for answers to social science 
questions. The researcher might be interested in how people deal with 
conflict in social relationships, how friendships develop between 
people who meet in chat rooms on the Internet, what communication 
is like between bosses and employees in a company, or how small 
groups make decisions. 

This technique has the advantage that the researcher can be guided, 
to some degree, by what other researchers and theorists have done. 
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For example, theorists on small group interactions have made catego- 
ries for different types of status in groups. Some people in a small 
group will have high status because they are experts on the subject the 
group is discussing. Other people will have status in the group be- 
cause they have a position in the group, such as the elected leader of 
the group. A researcher who is studying small group decision-making 
might look for evidence of these types of status and how they affect 
decision making. This approach can be both good and bad. The theo- 
ries might guide the researcher to find things that he/she might not 
otherwise notice. On the other hand, the researcher might have a hard 
time seeing new things, because he/she is only looking for types of 
status that had already been identified. 

Searching for Missing Information 

Sometimes what people do not say can be as revealing as what 
they do say. Therefore, a researcher might learn something from pay- 
ing attention to what people leave out. For example, in one study, a 
researcher was interviewing Chinese people about birth control in 
China. The researcher found that there were certain aspects of the 
government’s birth control policy that people would not comment on 
or discuss. She felt that interviewees were using this silence to protest 
those aspects of the policies. 

Interpreting missing information is a risky strategy, because there 
can be a variety of reasons for silence on a particular aspect of a sub- 
ject. Participants might be silent because they assume that the re- 
searcher already knows the information or is not interested in it. It 
might be a sensitive or embarrassing topic. The participant might not 
trust the interviewer, or might not want to talk about the subject with 
others present. It is even possible that the participant did not fully 
understand the questions. Therefore, the researcher needs a good fa- 
miliarity with both the topic and the participants to make inferences 
about the reasons for silence. 
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Conclusion 

Word-based techniques are ones which involve looking at the ac- 
tual words that participants in a study use. This is often a good way to 
begin looking at data. Scrutiny-based techniques involve looking at 
meanings that re-occur (or, in the case of missing information, ex- 
pected meanings which do not appear). 

Application Questions 

Think about the word-based and scrutiny-based techniques that 
you have learned in this chapter. How could you apply them to your 
data? 
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There are two other ways to approach data analysis in a qualitative 
approach. One involves looking at the linguistic features of the data. 
Another involves physically manipulating the data, by marking it, 
cutting it up and moving it around, etc. 



Sometimes the linguistic features that people use to describe their 
experiences are revealing, and one way of analyzing data is to look at 
these features. 

Metaphors, Similes, and Analogies 

Metaphors, similes, and analogies are figures of speech where two 
things that are different are compared to each other. Though the things 
that are being compared are different, they have some characteristics 
in common. If you say, “Life is like a journey,” life is being compared 
to a journey. Even though these two things are not really similar, they 
do have some qualities in common. Both life and a journey have a 
beginning and an end, for example. You can look at metaphors to try 
to understand the characteristics people believe something has. 

One researcher studied attitudes toward marriage by looking at the 
metaphors Americans used to talk about marriage. She found that many 
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of the interviewees used metaphors related to marriages being strong 
or weak. They talked about marriages being “as solid as a rock,” which 
means that the marriage was strong and solid, or about a marriage 
being “built on sand,” which means that the marriage was not strong. 

Transitions 

Another way to study the linguistic features of a language is to 
look at the changes in the content. For example, in a written text, 
starting a new paragraph can signal a change in the theme. In speech, 
pauses, changes in tone, or phrases like “on the other hand” or “to 
return to the main point” can be used to signal changes in theme. 

One area in which researchers have looked at transitions is in turn- 
taking in conversation. As two people talk, there are certain signals 
that one speaker is finished and the other speaker can begin speaking. 
For example, rising intonation, as in a question, can indicate that it is 
the hearer’s turn to begin speaking. 

Connectors 

Connectors are words or phrases that indicate the relationship be- 
tween two ideas. For example, in the sentence “I was late for class 
because I slept late,” what comes before “because” (being late for 
class) is caused by what comes after (sleeping late). 

Some relationships that might indicate themes include the follow- 
ing: 

1. attributes (A is B) 

2. contingencies (if A, then B) 

3. functions (A is a means of causing B) 

4. spatial orientations (A is close to/far from B) 

5. operational definitions (A is a tool for doing B) 

6. examples (A is an example of B) 

7. comparisons/contrasts (A is similar to/different from B) 

8. class inclusions (A is a member of category B) 
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9. synonyms (A is equivalent of B) 

10. antonyms (A is opposite of B) 

11. provenience (A is the source of B) 

Looking at how these connectors are used can help researchers 
identify themes. 

Tactile or Manipulative Techniques 

The last category is techniques that involve doing something physi- 
cal with the texts, such as marking them or moving them around. 

Pawing 

“Pawing” involves reading through the texts and marking key words 
and themes that seem obvious. You might use different colors for each 
theme that you notice. As part of this process, you might sort the pages, 
put them in file folders, put them in different orders, etc. You should 
probably read entirely through all the texts at least twice, and possi- 
bly more often. 

Unmarked Texts 

After you finish marking the texts with different colors for differ- 
ent themes, you will probably find that you have parts of the texts that 
are still not marked. You should look at the unmarked parts of the 
text. What themes do you see in the unmarked parts? You can look in 
the unmarked parts of the text to find themes that were not immedi- 
ately obvious to you. 

Cutting and Sorting 

Another way to manipulate the texts physically is to cut out what 
you consider the important parts of each text and put them on a card, 
along with identifying information about where the quote came from. 
Then you can sort the cards into different piles, with similar quotes 
together. You can name each of the piles of cards according to the 



134 Part HI Qualitative Research 



theme that they represent. You can divide themes that you find into 
sub-themes. If you are working with other researchers, you might each 
sort the cards independently. This might help you identify more themes. 

Choosing Analysis Techniques 

No one analysis technique is “best” for all situations. In most cases, 
it is best to combine two or more techniques. For example, you might 
begin with one of the word-based techniques to begin identifying 
themes. Once you have identified some themes, you might begin “paw- 
ing,” reading through the texts and marking these and other obvious 
themes. Next, you might use the comparison/contrast method by di- 
viding the texts between, for example, those produced by men and 
those produced by women, and see if you find different themes. Once 
you have identified the major themes, you might try cutting and sort- 
ing to find sub-themes of the themes you have identified. 

The type of text has some influence on the techniques that you use. 
For example, you can use word-based techniques for short answers to 
questions on a questionnaire, but you might not have enough material 
to use scrutiny-based techniques. The techniques that use linguistic 
features such as metaphor are best used when you have longer narra- 
tive texts. 



Conclusion 

There are a variety of techniques that you can use to analyze the 
data that you gather using qualitative approaches. You can use one 
technique to identify themes and then use other methods to identify 
more themes, or to find sub-themes within your themes. 

Application Questions 

How could you apply these techniques based on the linguistic fea- 
tures of the text or on physically manipulating the text? 
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Of the 12 techniques that you have learned about, which do you 
think you could combine to evaluate your data? In what order would 
you do them? 



PartrV 

Appendices 



Appendix I 

Writing Research Papers 



Research papers are usually divided into five major sections: In- 
troduction, Literature Review, Methods, Results, and Discussion/Con- 
clusions. These are usually the headings given to these sections, though 
sometimes they are different. As a writer, you must know what be- 
longs in each section. In addition, if you read research papers, it is 
helpful to know what to look for in each section and how it fits in with 
the paper and the study as a whole. 

Parts of the Research Paper 

Introduction 

The introduction briefly informs the reader of what the study is 
about and why it is important. It should not go into great detail, but it 
should give the reader an idea of what to expect. The introduction 
should tell in a few sentences what is covered in the literature review 
and what the research deals with. 

The introduction is not necessarily labeled with a subheading, be- 
cause it comes first, and so it is obvious that it is the introduction. It 
may be combined with the literature review. 

Literature Review 

A literature review is a section which summarizes what other schol- 
ars have said about your subject. (In this meaning “literature” does 
not mean novels. It means writings about research, theory, etc., re- 
lated to a specific topic.) The literature review provides information 
about research and theory related to the research being reported in the 
paper. You need to read books and papers about the subject you are 
interested in and briefly summarize the ones that are related to the 
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research you are doing. The literature review also presents the spe- 
cific problem that the study deals with, either in the form of hypoth- 
eses or research questions. In this section, you include answers to 
some of the following questions: 

Why is this topic important? 

What have theorists said about it? 

Has there been any other research in this or related areas? If so, 
what were the results? 

For example, if you want to do a study on the development of 
relationships between people who meet on the Internet, you should 
read books and research papers about the development of relation- 
ships in general and anything you can find about how relationships 
develop on the Internet. When you write your literature review, you 
include some general information as background, and also some spe- 
cific information about the particular area that you are interested in. 

This information helps the reader understand why your study is 
important and where it fits into research that has already been done or 
theories that have been developed. In some cases, you may feel that a 
previous study had problems. If so, you should discusses the prob- 
lems and how they may have influenced the results of that study. For 
example, imagine that you read a study on the development of Internet 
friendships which only used subjects who were teenage girls. You 
might argue that these friendships were not necessarily typical of all 
Internet friendships. They may be different from relationships formed 
by teenage boys or people of other ages. You would particularly do 
this if you were going to report on a study which included subjects of 
different ages. 

If you are using terms with specific meanings, you should also 
define them in the literature review. Some expressions have been de- 
fined in different ways, and you need to explain which definition you 
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are using. You might use a definition that some other researcher or 
theorist has used, or you might use your own definition. If you use 
your own, you might need to explain why yours is different from, and 
better than, previous ones. For example, if you are doing a study of 
“Internet friends,” you might define Internet friends as “people who 
had their first contact through the Internet, who self-describe them- 
selves as friends, and who have never meet face-to-face.” 

At or near the end, you express the hypotheses (statement of the 
relationships between variables) that are being tested or the research 
questions that you are attempting to answer in the study. The informa- 
tion in the literature review should provide arguments based on theory, 
results of previous studies, or logical arguments, for the hypotheses 
that are being studied, or background for the research questions if the 
study is exploratory. 

Methods Section 

In the methods section, you describe what you did in the study. 
Ideally, you should provide enough information for a reader to carry 
out the same study. Due to space limitations, this may be difficult, but 
you should still include enough information that the reader can ana- 
lyze your method and decide whether the results of a study done with 
that method are valid. The section includes information about who 
participated in the study, what materials, measures, etc., were used, 
what the participants were asked to do, and what analyses were per- 
formed on the data. The methods section is divided into several sub- 
sections, depending on what information is necessary. 

Overview/Research Design. In the overview or research design 
subsection, you briefly describe the method used. Included is infor- 
mation about what is being compared in the study, what materials or 
measures are used, and so on. This section should be short, not more 
than a paragraph or two. Its purpose is to give the reader just a brief 
overview of the study so that the significance of the rest of the infor- 
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mation in the methods section is easier to understand. 

Subjects/Participants, In the subjects or participants subsection, 
information should be included about subjects in the study, including 
who they were, how they were chosen, how many were involved (both 
the total number and the number assigned to each experimental con- 
dition; if males and females are considered separately, their numbers 
should be reported separately as well), and under what circumstances 
they were studied. Any other information that is relevant to the study 
should also be included, such as age, sex, level in school, language 
proficiency (for non-native English speakers), and socio-economic 
status. A characteristic that is a variable in the study should be de- 
scribed in detail. Information about whether any participants dropped 
out of the study and why they dropped out should also be included. 

It is particularly important that you explain how the subjects were 
chosen. In some cases, the subjects in a study were asked to volun- 
teer. In other studies, an intact group, such as an existing class, is 
used. In other cases, the subjects are chosen randomly from a larger 
group. Each of these situations has the potential to have an effect on 
the outcome of the research. Therefore, readers need to know about 
them. 

Materials/Measures/Equipment. In the materials or measures sub- 
section, you describe any materials that you used in the study, includ- 
ing stimulus materials (for example, if you have subjects read a read- 
ing passage and respond to it, this is stimulus material) and measure- 
ment instruments (for example, a questionnaire). You should briefly 
describe the stimulus materials, measurement instruments, etc., in the 
materials section and explain how they were written and developed. 
If you used persuasive messages, a description of the messages and 
how they were different should be included in the materials section. If 
you use a questionnaire, interview protocols, rating scales, and so on, 
you should describe what kinds of questions or other measurements 
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you used and what you are attempting to measure with those ques- 
tions. 

If you used materials that were originally developed by someone 
else, or if you modified someone else’s materials, you should credit 
that person and include a reference to the publication where you found 
those materials. (Unless you are using a well known measurement 
instrument, you should include the measurement instrument in the 
appendix of your paper.) 

In addition, if any equipment is used for the study, the equipment 
and its function in the experiment should be described in the equip- 
ment subsection. 

Procedures. In the procedures subsection, each step in the study is 
described. You should specify what instructions were given to the 
participants, how groups were formed, and what the manipulations 
were. Procedures such as randomization (assigning participants to 
groups at random) and blind coding (keeping the assistants who rate 
participants, etc., from knowing what the hypotheses were or which 
group the participants were in) should also be included. 

Analyses. The statistical procedures that you used are described in 
the analyses section. 

Importance of the methods section. The methods section often 
does not seem to be very important, and readers may skip it or skim 
over it very quickly. However, this section is crucial, because it al- 
lows readers to evaluate whether there are problems with the methods 
used, that is, whether the methods used allow threats to validity. You 
can find out the results of the study by reading the results section, but 
you cannot evaluate what the results really mean without understand- 
ing how the information was gathered. 

Results 

In the results section, the data is summarized and the results of the 
analyses are reported. These may be in prose form or in the form of 
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tables and figures. The amount of detail that is provided in this sec- 
tion should be enough to show why the hypotheses were either ac- 
cepted or rejected, or to show how the research questions should be 
answered. 

Discussion/Conclusion 

The final section of the paper itself is the discussion or conclu- 
sions. There is usually a summary of the results in non-technical terms 
and an explanation of what those results mean and the conclusions 
that the research draws from them. As a writer, you should be careful 
that the conclusions that you draw can actually be supported by your 
results. This section should include the answers to the original ques- 
tions raised in the introduction/literature review. The broader impli- 
cations of the results may also be discussed and perhaps further ques- 
tions that may be raised by the results of the study. 

There may be a subsection on the limitations of the study in this 
section. In this subsection, the research explains limitations on the 
generalizablity of the study. For example, if all of the participants 
were college students, the writer might point out that other age groups 
may be different. The writer should also point out threats to internal 
and external validity of the study and how these threats might have 
influenced the results. 

Other Sections 

In addition, to the body of the paper, there are a few other sections. 

List of References. The sources that are cited are listed in the list 
of references. 

Appendices. The appendices include information that does not fit 
into the main body of the paper. It may include detailed descriptions 
of a procedure or a questionnaire. 

Notes. Notes are brief explanations, such as expressions of appre- 
ciation, notices of support from grants, etc. 
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Checklist 

As you write your paper, keep the following points in mind for 
each section: 

1 . Introduction 

a. Inform readers what study is about and why it is important 

b. Should be brief 

2. Literature Review 

a. Summarize what other scholars have said about the subject 

b. Include information about why topic is important, what theorists 
have said, what research has been done 

c. Show how your study fits in the topic area 

d. Define terms with specific meanings 

e. Include hypotheses or research questions 

3. Methods 

a. Describe what you did in the study 

b. Give enough detail that someone else could carry out the study, 
if possible 

c. Include overview, and information on subjects/participants, ma- 
terials/measures/equipment, procedures, and analyses, 

4. Results 

a. Summarize the results 

b. Provide enough detail to justify conclusions 

5. Discussion/Conclusion 

a. Summarize results in non-technical terms 

b. Be certain that the conclusions are supported by the results 

c. Include limits on generalizibility and threats to internal and ex- 
ternal validity 

6. Other sections 

a. List of References 
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b. Appendices 

c. Notes 

Discussion Questions 

According to this chapter, there are five sections in a research pa- 
per. Read the research paper in Appendix III. Identify each of the 
sections and subsections described in this chapter. What characteris- 
tics of each section described in this appendix do you see in the pa- 
per? 



Appendix II 

Citing Sources and Avoiding Plagiarism 

If you use someone else’s ideas or work in your paper, and particu- 
larly if you quote someone else directly, you need to acknowledge 
that person. In this chapter we will discuss what you need to make 
references for, how to make references, and how you can avoid pla- 
giarism. 



What is Plagiarism? 

Plagiarism is using someone else’s words, ideas or other kinds of 
work without permission or acknowledgement. For example, if you 
discuss someone else’s theory in your literature review, you need to 
give credit to the person who developed the theory. If you do not give 
credit to that person, the theory appears to be your own, and you are 
committing plagiarism. Another example would be if you copied the 
exact words from someone else’s paper and used it in your own paper, 
without putting quotation marks around the words and indicating where 
the words came from. 

Plagiarism may not be taken seriously in some cultures, but it is 
taken very seriously in Western cultures. Plagiarism involves stealing 
someone else’s ideas, expressions and work. If you do it for a class in 
a British or American university and your teacher finds out, you will 
almost certainly fail the class, and you may be expelled from the uni- 
versity, as you would if you cheated on an exam. If you use plagia- 
rism in a paper which you submit to a US or European journal, you 
will also be in serious trouble. 

In addition to using someone else’s ideas, work, or words without 
giving them credit, plagiarism includes buying, stealing or borrowing 



II. Citing Sources and Avoiding Plagiarism 147 



someone else’s paper, or hiring someone to write your paper. 

Since it is very easy to copy from the Internet resources, plagia- 
rism has become a more serious problem in recent years. For non- 
native speakers of English, it is not easy to paraphrase passages, and 
they sometimes violate plagiarism rules without intending to. Stu- 
dents who write academic English should know what plagiarism is 
and how to avoid it. 



How to Avoid Plagiarism 

It is obvious that you should not use someone else’s pictures, pho- 
tos, charts, drawings, tables, etc., as they are, as part of your own 
work. If you want to do that, you need to receive permission to do so 
from the owner, and also you need to indicate that you received per- 
mission and from whom you received the permission. If you just, for 
example, quote some figures from a table or chart in your review of 
literature or discussion, you do not need to receive permission to do 
so, but you need to cite which source you used. 

If you are directly quoting from someone else’s work, or if you are 
referring to someone else’s ideas or theories, you need to include a 
citation that tells whose ideas or words they are, and where they came 
from. You do that by putting their name(s), the year of publication, 
and the page number in parentheses within the paper. At the end of the 
paper, you have a “References.” In the References, in alphabetical 
order by the author’s last name, you provide information about the 
publication or oral presentation where you got the words or the idea. 
In the social sciences, teachers generally require you to use the for- 
mat from the American Psychological Association (APA), which tells 
you how to make citations. Most journals in the social science use 
this format, or a similar format based on it. 

If you use any direct quotes — that is, the exact words in the origi- 
nal source — from a magazine, book, paper, newspaper, song, TV 
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program, movie, Web page, computer program, letter, advertisement, 
or any other medium, you need to put the quote in quotation marks (if 
it is less than three lines long) and cite the source. If the direct quota- 
tion is more than three lines long, you should leave blank lines before 
and after the quotation and indent the lines five spaces. In that case, 
you do not need quotation marks. You should not quote more than a 
short paragraph from the copyrighted material without permission. 

For example, if you were writing about intercultural communica- 
tion, you might write the following; 

As Kitao & Kitao (1995) have explained, 

To a great extent, our perception is influenced by our cul- 
ture. For example, the characteristics that determine beauty 
vary in different cultures. Centuries ago in Japan, women 
blackened their teeth, and this was considered beautiful. 
Today, perceptions of beauty have changed, and that is not 
considered beautiful, (p. 91) 

In the list of references at the end of the paper, you would see: 

Kitao, S. K. & Kitao, K. (1995). Communication Theory. Tokyo: 
Gakushobo. 

(You can find more information about how to cite sources at web 
pages linked to; 

http://ilc2.doshisha.ac.jP/users/kkitao/online/www/referenc.htm#apa.) 

If you use ideas from another writer, you need to cite the source. 
You cannot present someone else’s ideas as if they were yours. Even 
if you get ideas from others through conversations or personal letters, 
you need to cite the source. 

If you were to paraphrase an idea, you would write: 
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“Rules of use” tell us, of all the sentences that are grammatically 
acceptable, what sentences are socially acceptable (Hymes, 1974). 
For example, the sentence “How old are you?” is grammatical, but it 
is not polite in most social situations in the US. 

In this case, Hymes, not the author of this paper, developed the 
idea of “rules of use.” Therefore, this author cites the source of the 
idea of rules of use in the list of references. 

Hymes, D. (1974). Foundations of sociolinguistics: An ethnographic 

approach. Philadelphia: University of Pennsylvania Press. 

If you paraphrase a passage, you do not need to put quotation marks, 
but you still need to cite the source. Plagiarism involves not only copy- 
ing wording but also stealing ideas. 

Exceptions to Rules about Plagiarism 

When you write your own experiences, thoughts, ideas, observa- 
tions, opinions, etc., you do not need to cite anything since they are 
your own. 

When you use common knowledge, that is, shared information or 
beliefs in your field of study or culture, you do not need to cite the 
source. When you mention some facts that many people already know, 
you do not need the citation. 

How can you judge whether the information is common knowl- 
edge or not? If you find such information in various sources without a 
citation, it is probably common knowledge. If your readers are al- 
ready likely to know it, or if they can find such information in general 
reference books, you can assume it is common knowledge. 

Even if what you are writing is common knowledge, if you quote 
someone else’s wording, you need quotation marks and a citation, of 



course. 
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When you write about the results of your observations or experi- 
ments, you do not need to cite anything, though you need to explain 
clearly how you arrived at the results. They are your own informa- 
tion. 



How to Make Sure to Avoid Plagiarism 

When you are taking notes from sources, including conversations 
with people or lectures by professors, make a distinction between that 
which is quoted and that which is your own idea. This is very impor- 
tant when you take notes for writing academic papers. In writing a 
paper, you need to put quotation marks around direct quotations, or a 
reference and a note if you just refer to the information. In either case, 
you need to put the source in the list of references. 

As you paraphrase or summarize, do not read directly from the 
original source. Read the source, write the ideas from memory, and 
then check your writing against the original for the accuracy of the 
content and to make certain you did not use the wording of the origi- 
nal passage. Cite the source, if the information is not common knowl- 
edge. 

When you quote something directly, you should put it in quotation 
marks. You should not include direct quotations very often, except 
when the original source has particular impact. In many cases, indi- 
rect quotations are better because they are smoother to read. In either 
case, you need to cite the source. 

Conclusion 

It is very important to know the rules under which you use infor- 
mation in a paper. It is easy to copy someone else’s ideas, words, or 
work, and it is possible to do so by accident. This is considered steal- 
ing, and it is serious. You have to be very careful to avoid it in your 
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academic work. It does not make your work less valuable if you give 
credit for the ideas and words. In fact, that makes your writing more 
objective and reliable. Be sure to include citations and avoid plagia- 
rism in your academic writing. 

Note: You can find more information about plagiarism and how to 
avoid it on web pages linked to http://ilc2.doshisha.ac.jp/users/ kkitao/ 
class/material/academic.htm. 

Discussion Questions 

Look at the paper in Appendix III and answer the following ques- 
tions: 

1. Does the paper include any direct quotations? If so, how does 
the author give credit for the quotations? 

2. Does the writer refer to anyone else’s research, information, 
ideas, or theories? If so, how does she give that person credit? 

3. Look at the list of references. What does the writer include (books, 
articles, etc.)? 
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Appendix III 



Preparation for and Results of a Short-Term 
Overseas Study Program in the United States 



S. Kathleen Kitao 
Doshisha Women’s College 



Abstract 



In the summer of 1992, 
Doshisha Women’s Junior College 
students went to Mary Baldwin 
College for a short-term overseas 
program for the first time. The stu- 
dents had an orientation class in 
Japan and spent five weeks in the 
US, where they attended classes, 
traveled and had homestays. 

In this paper, I looked briefly 
at other studies of students going 
overseas and how the experiences 
affected their views of the country 
they went to. Before and after go- 



ing to the US, the students filled 
out a questionnaire about their 
English proficiency, their opinions 
of the US, and their opinions of the 
orientation. The results indicated 
that they felt that their English pro- 
ficiency had improved. Their opin- 
ions of the US became generally 
more positive, and more detailed. 
I discussed these results and the 
students’ own impressions from 
their journals. The appendices in- 
clude a detailed description of the 
orientation program. 



Doshisha Women’s College has been sending students overseas 
for a summer program since 1982. At that time, students of the four- 
year college went to Mary Baldwin College in Staunton, Virginia, 
and traveled in the United States. After the junior college was estab- 
lished in 1986, a summer program was established for junior college 
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students with Warwick University in Britain. In 1992, junior college 
students went to both Mary Baldwin College and Warwick Univer- 
sity for the first time. 

In this paper, I look briefly at other studies of students going over- 
seas. I discuss the Mary Baldwin College program for the junior col- 
lege students in 1992 and a study of the effects of the program on 
students’ language proficiency and their image of the United States, 
and the students’ comments about their experiences. Finally, I discuss 
suggestions for future programs from the students. 

Literature Review 



Overseas Study 

Although many Japanese students go overseas for study, either short 
term or long term, there has been relatively little study of these stu- 
dents, their preparation, or the results of the students’ experience over- 
seas (Yashima and Viswat, 1991). 

It is difficult to compare the studies that have been done of Japa- 
nese students going overseas, because of the widely different popula- 
tions and situations that they involve. Studies that have been done 
include Yashima and Viswat (1991), which looked at changes in the 
image of the United States for Japanese high school students who 
spent a year studying in the United States; and Higuchi, Saito, 
Lamarche, Shelangouski, and Kikuchi (1982) which looked at junior 
college students who were in Canada for a homestay experience. In 
both cases, the image that the students had of the host country dete- 
riorated, and after their experience, they considered the people of the 
host country less friendly, less warm, less polite, less reliable, less 
kind, etc. However, in spite of being somewhat less positive, the par- 
ticipants’ images of the host countries were still relatively positive, 
and the researchers concluded that the students’ images had become 
somewhat more realistic through their experiences. 
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A study of a short-term program by Nozaki (1987) indicated that 
students’ image of Americans became significantly more positive af- 
ter a short stay in the United States on 14 out of 28 items: honesty, 
generosity, intellectual curiosity, sense of responsibility, warmheart- 
edness, impartiality, diligence, kindness, prudence, politeness, clean- 
liness, morality, trustworthiness, and peaceableness. Students’ images 
did not become more significantly negative on any item. Nozaki also 
looked at changes in the students’ images of Japan, but found no sig- 
nificant differences. 

Overseas Orientation Courses 

In sending students overseas, no matter what their country of ori- 
gin and host country, orientation is extremely valuable. Studying over- 
seas is “potentially one of the most broadening and maturing educa- 
tional experiences that a young person can have.” However, “merely 
sending young people to live abroad for a while does not guarantee 
that they will learn a great deal more than if they remained at home” 
(Grove, 1989, p. xi). While Grove was referring to long-term pro- 
grams, orientation is also important to the success of short-term pro- 
grams. 

The orientation that students received and the program at Mary 
Baldwin College are described in Appendix A. 

Research Questions 

1. What kinds of preparation are useful for students planning to go 
overseas? 

2. How did a short-term overseas program in the US affect Japa- 
nese students’ English language proficiency? 

3. How did a sort-term overseas program in the US affect Japanese 
students’ impressions of the US? 
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Methods 



Overview/Research Design 

In this study, I looked at the effects of a short-term overseas pro- 
gram on students’ language proficiency and impressions of the United 
States. This was done using a questionnaire administered at the end of 
the 3-month orientation and again after the students’ return to Japan, 
and also through an analysis of students’ journals. 

Subjects/Participants 

The participants in this study were 33 Japanese female junior col- 
lege students in a short-term overseas program. All of the students 
took part in the study as a requirement for the orientation class. They 
were chosen by interview from among 115 applicants for the class, 
based on their English language proficiency and the attitudes they 
expressed toward overseas study. They were all 18 or 19 years old. 

Materials/Measurement Instrument 

Students filled out a questionnaire (see Appendix B) in either Eng- 
lish or Japanese at the beginning of the orientation program in May 
1992 and in October 1992, after they had returned to Japan. Students 
were asked about their previous experience overseas and about how 
they prepared for the overseas program. They were also asked to rate 
their English proficiency on a scale of 0 to 10. 

Students were asked to provide five positive and five negative ad- 
jectives to describe the US. This method has both advantages and 
disadvantages in comparison to the method used by Yashima and 
Viswat (1991), Nozaki (1987), and others, in which participants rated 
their host country on a semantic differential, using adjective pairs such 
as friendly/unfriendly, kind/unkind, and dependent/independent. While 
this method provides clear and easily quantifiable results, the results 
are in terms of what the researcher, not the participant, considers most 
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important. Asking students to provide their own adjectives makes the 
results more difficult to analyze, but it does provide more access to 
what is uppermost in the minds of the participants themselves. In ad- 
dition, this instrument was more sensitive to some qualitative rather 
than quantitative changes. The students’ image of the United States as 
a dangerous country before and after the trip, described below, would 
be an example of this advantage of this instrument. 

In the post-program questionnaire, they were asked for sugges- 
tions for future programs. 

Procedures 

In addition to answering the questionnaires, the students were re- 
quired to keep a journal about their experiences during the orientation 
program and while they were in the US. They wrote a minimum of 
one page per week during the orientation and two pages per week 
during the five weeks in the US. Students were instructed to write 
about their preparation, their experiences in the program, and how 
they reacted to their experiences. 

In addition, Americans who worked with the students were inter- 
viewed about the students’ proficiency. 

Journals were kept by the students and turned in in October, after 
they had returned to Japan. 

Analyses 

In order to answer the question about improvement in language 
proficiency, students self-ratings before and after the program were 
compared. A t test was performed to compare the pre-program and 
post-program ratings. 

The researcher read all of the students’ journals, paying particular 
attention to the students’ impressions of the US and Americans and 
their emotional reactions to their experiences. Students’ journals are 
quoted in the paper to illustrate or explain the results of the study. 
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Results 

Reasons for Joining the Program 

When asked for their reasons for wanting to enter the MBC pro- 
gram, students answers included a desire to: 1) speak to foreigners, 2) 
learn English (abroad) or improve English proficiency, 3) participate 
in a homestay program, 4) communicate/exchange views with Ameri- 
cans, 5) experience American education, 6) prepare for future study 
at an American university, 7) teach Americans about Japan, 8) learn 
about American culture and life, 9) see how Americans live, 10) ex- 
pand their world, and 11) learn American pronunciation. The most 
frequently mentioned reason was a desire to participate in the homestay 
program. 

Students’ Perceptions of Their Preparation 

Students were asked on the questionnaire which types of prepara- 
tion they had found most and least useful. (Descriptions of the texts 
and other preparations and how they were used are included in the 
description of the program in Appendix A.) The students’ responses 
indicated that learning about functions (e.g., requests, apologies, com- 
plaints; Communicating with Americans) was the most useful. Seven- 
teen students responded that it was the most useful type of prepara- 
tion. Learning about traveling and using English {Communicating in 
the U.S.) was listed by ten students. Presentations by other students (5 
students) and learning about American culture {Background to the 
USA) (2 students) were also mentioned. As for the least useful prepa- 
ration, Reading about American culture {Background to the USA) was 
perceived as being the least useful. It was mentioned by the largest 
number of students (15). Presentations were mentioned by eight stu- 
dents. Kanko kara business made: Travel eikaiwa (4 students) and 
Communicating with Americans (1 student) were also mentioned. Five 
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students specified that they did not consider any of the types of prepa- 
ration “least useful.” 

Increase in Students’ English Proficiency 

In the questionnaire, students were asked to rate their English pro- 
ficiency on a scale of zero to ten when they began the program in May 
and again after they returned from the United States. Before the pro- 
gram, the mean rating of their proficiency was 3.56, with a standard 
deviation of 1.16. After the program, the mean was 5.74, with a stan- 
dard deviation of 0.93. I did a one-tailed t test comparing the two 
values, which indicated that the increase was significant (t = -10.73; p 
< .05). 

In addition to asking the students to rate their proficiency, I inter- 
viewed the five counselors about their qualitative perceptions of the 
increase in the students’ English proficiency. This interview was con- 
ducted ten days after the students arrived at MBC. The counselors 
were all able to see clear improvements in the students’ English profi- 
ciency. Specifically, they said that they had gone from speaking in 
one- to two-word utterances to full sentences. Students’ comprehen- 
sion also improved in that the students were capable of understanding 
English spoken at a nearly natural speed. When the students had first 
arrived, the counselors had felt it necessary to slow their rate of speak- 
ing considerably. The counselors felt that the relaxed atmosphere of 
the program was conducive to building confidence. 

Changes in Students’ Image of the United States 

In order to assess changes in the students’ images of the United 
States and Americans during the orientation and their time abroad, 
students were asked to list five positive and five negative adjectives 
describing the United States on the pretest and posttest. Students re- 
sponded with up to five adjectives (or frequently nouns or phrases) 
for the positive and negative lists. One student on the posttest did not 
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provide any negative response. 

Positive images. The most common adjective used to describe 
Americans or the United States was “friendly,” and this was an image 
that became stronger during the period of the study. Seventeen of the 
students described Americans as “friendly” in the posttest, up from 
seven on the pretest. Related adjectives were “kind”, which appeared 
on 14 posttests but only three pretests; large- or big-hearted (three 
each on the pretest and posttest); generous (twice in the posttest and 
once in the pretest); and gentle (once on the posttest). Three students 
on the posttest, but none on the pretest, noticed politeness to women. 
Americans were also described as being active (five on the pretest 
and five on the posttest) and cheerful (eight on the pretest and eight 
on the posttest) and people who express their opinions openly (3 stu- 
dents in the posttest and one in the pretest). Some students perceived 
Americans’ lifestyle as being leisurely (three students in the posttest), 
though this was sometimes perceived as being negative in that clerks 
were slow (four students in the posttest). 

As for the country itself, the image of the United States as being 
large seems to be the strongest image, both on the pretest (19 stu- 
dents) and posttest (26 students). It was also described as beautiful (4 
students in the posttest) and with a lot of greenery and natural scen- 
ery, even in cities (4 students on the posttest). 

Negative images. In the pretest, students’ negative images of the 
United States were largely related to broad social problems. These 
were listed less often in the posttest. For example, the social problems 
that were listed included racial discrimination (13 in the pretest and 1 
in the posttest), crime and violence (8 in the pretest and 4 in the 
posttest), and AIDS (3 in the pretest and 1 in the posttest). 

Students saw the United States as dangerous both before and after 
the program (13 in the pretest and 15 in the posttest). Interestingly, 
however, the adjective “dangerous” was never qualified in the pre- 
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test, but in the posttest, it was usually qualified by such phrases as, “in 
some places,” “mainly New York,” and “large cities.” 

How students felt their images had changed. The students were 
also asked directly how their image of the United States had changed 
as a result of the time they spent there. The replies of all the students 
indicated that their images were more positive than or the same as 
they were before the trip. None of the students indicated that their 
image was more negative. Thirteen of the students indicated that the 
United States was safer or less dangerous then they had expected. 
Four of the students indicated that they felt “closer” to the United 
States or Americans than they had before the trip. 

Discussion and Conclusions 

Students’ Perceptions of Their Preparation 

Many of the students clearly felt that learning about functions in 
English was useful to them during their time in the United States. 
Learning about the situations they would be in as travelers was also 
considered beneficial. Probably these were considered useful because 
they were very specific and concrete, so students could clearly see 
their usefulness. 

Learning specific expressions for travel situations {Communicat- 
ing in the U.S.) was not considered the most useful preparation by any 
student and five students considered it the least useful preparation. 
However, because little emphasis was placed on this (there was no 
specific homework, other than to read the expressions, and they were 
only used in role plays at the end of the course), students may not have 
spent sufficient time on this type of preparation to benefit from it. 

Learning about American culture, at least from Background to the 
USA, was not perceived as being particularly helpful. However, it is 
well known that a knowledge of culture is necessary to effective com- 
munication (Kitao, 1977), and that background knowledge influences 
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comprehension (Anderson, 1978; Kitao, 1989). For example, one stu- 
dent mentioned that she wished she had known more about American 
government, since her host mother talked about it, and a few other 
students mentioned that it would have been helpful to know more 
about various aspects of American culture, including daily life. It may 
be that because the students can see a direct use for functions or ex- 
planations of situations they will be in as travelers, they can easily 
recognize the usefulness of such information. However, many of the 
students may not have been as aware of the usefulness of background 
knowledge in communicating with English speakers. Since this type 
of preparation is not as concrete or obviously useful as learning about 
functions or situations, students may not have been aware of its use- 
fulness. 

Increase in Students’ English Proficiency 

Clearly, the students perceived that their English had improved 
noticeably, and this was supported by the results of the interviews 
with their counselors. It is unlikely, however, that in three weeks of 
classes and two weeks of travel and homestay they learned a great 
deal more about English granunar or vocabulary. It is more likely that 
what actually improved was their ability to make use of the knowl- 
edge of English that they already had. 

In addition to gaining confidence in their English, many of the 
students seemed to have gained more motivation to further improve 
their English. Many of them commented in their journals that when 
they returned to Japan, they wanted to work on their English profi- 
ciency so that they could conununicate better when they returned to 
the United States. 

Oiauges in Students’ Image of the United States 

The students’ image of Americans was made more positive by their 
experience in the US. They found Americans kind and friendly. This 
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may have been, in part, because they spent the most of their time in 
the States in a small town (Staunton, Virginia, population 20,000). 
Also, they were only in the US for five weeks. Previous studies of 
longer-term overseas students indicate that students’ view becomes 
less positive and more realistic over a longer period of time. Students 
may have still been in a sort of “honeymoon” period at the end of five 
weeks. 

The reduction in the number of students who listed racial discrimi- 
nation as a factor in their negative image of the United States was 
particularly noticeable. Though students had heard about discrimina- 
tion through the media before going to the United States, and some 
expressed concern about “Japan bashing,” they experienced little or 
none of it themselves, so this image changed. In fact, to my knowl- 
edge, only one student may have experienced any racism. While at 
Disneyland, she thought (but was not certain) that she overheard some- 
one say, “Jap.” 

Students appeared to have learned that different places in the United 
States varied in how dangerous they were. In her journal, one student 
wrote, “I came to like America after this program. I changed my thought 
about America. Before I visited America I thought that America was 
only dangerous, but it is not true. Some places are dangerous, but 
there are a lot of nice places. And people were very kind and friendly.” 

Most of the students, either in this questionnaire or in conversa- 
tion with me, their teachers or their counselors, expressed interest in 
returning to the United States, possibly even to live or study. Three of 
the students indicated that they had not been particularly interested in 
the United States before this trip, but now they were very interested in 
it. (One of them wrote, “At first I wanted to go to England, but I think 
I’m lucky girl, I couldn’t go to England. My favorite country is 
America.”) Clearly, the image of the students was more positive than 
it had been before the trip. 
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Students’ Advice for Future Programs 

The students were also asked about their advice for the directors of 
future programs and what they wished they had known before going 
to the United States. No single suggestion was made by more than 
four students, and most suggestions were made by only one student. 

Suggestions for the preparation before going to the United States 
included: learning more about American culture, including lifestyle, 
customs, and history (particularly related to places students visited, 
perhaps in Japanese); learning more about MBC (for example, how 
many students are there, what majors are offered, etc.); more oppor- 
tunity to speak to foreigners before going to the United States; more 
study of English language, particularly conversation, vocabulary, and 
pronunciation; and more emphasis on explaining Japanese culture in 
English. 

Suggestions for the portion of the program at MBC included more 
free time for rest, class preparation, or writing in journals; more time 
for preparation for Japan Night; a longer time spent in classes, per- 
haps after homestay; more counselors, perhaps one counselor per three 
to four students, because students speak English most with counse- 
lors; and more opportunity to talk with teachers and other Americans. 

Students were also asked about their favorite and least favorite 
city. Of the cities the students visited after leaving MBC, New York 
and Los Angeles were the most popular, though all of the cities were 
mentioned as a favorite by at least one student. Most of the students 
said that there was no city that they did not enjoy visiting. A few 
students, however, mentioned that the schedule had been difficult. 

Conclusion 

The students obviously felt that they had benefited from their ex- 
perience of studying overseas. They felt that they had been broadened 
by the experience. As one student wrote (after having commented 
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that she had gained weight in the United States), “I may be over- 
weight, but I not only my body grew but also my heart. I never forget 
my American life.” Another wrote, “I could know American idea, 
American history and American culture, and we could tell Japanese 
history and culture in return. I could feel close to America. I love 
America and I want to visit America again.” Another student felt that 
the experience had had an especially profound effect on her. She wrote 
after the time at Mary Baldwin College, “In this three weeks I found 
my mind changed. I found there was nothing I can’t do. I want to 
challenge everything without fearing. I’m sure if I hold out, my dream 
will come true.” Another student wrote, “Through this trip, I feel I 
grew up.... I learned many things. Probably I come to be able to speak 
English better than before I went to America.... I’ll never forget this 
trip.” They perceived that their English had improved, they had more 
motivation to study English, their image of the United States and of 
Americans was more positive, and they had more confidence in them- 
selves. 

Future studies of these topics would be worthwhile. For example, 
more in-depth studies of what students found useful and, perhaps more 
important, why they found it useful, would help in the future prepara- 
tion of students for overseas study. (Studying what students found 
useful might be more effectively done with a Likert scale or a seman- 
tic differential.) In addition, studies on the lasting effects of the expe- 
rience and how much changes in students’ attitudes toward and im- 
ages of the host country persist would be valuable. 
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Appendix A 

Preparation for Going Overseas 

Choosing Participants 

Students were chosen in late April 1992 for the Mary Baldwin 
College program from a pool of 115 applicants from Doshisha 
Women’s Junior College for the MBC program and the program at 
Warwick University. Students were chosen based on questions in Eng- 
lish and Japanese about themselves and their goals for going over- 
seas, a multiple-choice quiz about American history and culture, and 
an interview. In the interview, which was conducted in both English 
and Japanese, students were rated according to their English profi- 
ciency, their goals for the program, and how responsible they seemed 
to be. 

Thirty-four students were chosen by this process. All of the stu- 
dents were first-year students in the junior college, ranging in age 
from eighteen to twenty years old. Six were Japanese majors and the 
remainder were English majors. One student had lived in Malaysia 
for three years, one had been in the United States for a 25-day homestay, 
and three had been overseas on vacation for less than one week. One 
student dropped out of the program in June and was not replaced, so 
thirty-three students participated in the program and made the trip. 

The Overseas Orientation Course 

The orientation class met twelve times between early May and late 
July, including an overnight session at a hotel. It was taught by one 
full-time faculty member, with the assistance of a staff member. The 
importance of attendance was stressed, and there were only two ab- 
sences during the course. Several types of preparation were combined 
in the orientation. 
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Textbooks. There were two textbooks, Communicating with Ameri- 
cans: Functions in English (Kitao and Kitao, 1991) and Background 
to the USA (Musman, 1982). In addition, two books. Communicating 
in the U.S.: English for Situations (Kitao, S.K. and Kitao, K., 1993) 
and Kanko kara business made: Travel eikaiwa [From Sightseeing to 
Business: Travel English Conversation](Kitao, K. and Kitao, S.K., 
1993), were used. I will discuss each of these textbooks individually 
and how they were used. 

Communicating with Americans takes a functional approach to the 
teaching of English. There are chapters on such functions as greeting, 
complimenting, agreeing and disagreeing, requesting, complaining, 
and expressing appreciation. Students prepared chapters in advance 
and did role plays with classmates in class. Background to the USA is 
a general book on American culture. Students were assigned to read 
and answer short answer questions about two chapters a week. Com- 
municating in the US. and Kanko kara business made: Travel eikaiwa 
organized around situations that travelers in the United States face. 
These situations include being on an airplane, in a post office, or at a 
hotel. In Communicating in the US., each of these situations is ex- 
plained along with the behaviors expected in them. Kanko kara busi- 
ness made: Travel eikaiwa addresses the same situations as Commu- 
nicating in the U.S., providing expressions and sample dialogues re- 
lated to those situations. Students were assigned to read through the 
expressions and dialogues, though these were not used directly in class 
until role plays were done in the last class. 

Student group presentations. Groups of students did presentations 
on various useful topics — packing and traveling, health and safety, 
talking about Japanese culture in English, American education, 
homestay, and so on. The groups also did presentations on each of the 
places they would be visiting — Virginia, New York, Boston, Niagara 
Falls, Los Angeles, and Washington, D.C. 
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Other. In addition, students saw videos in class about two of the 
places they would be visiting during their time in the US. They were 
also given handouts with information about various practical aspects 
of the trip, for example, how to act during their homestay. There was 
also an opportunity to see a video from last year’s program at MBC 
and to ask questions of last year’s participants. 

Journals. Students were required to keep journals during the ori- 
entation and to continue to keep them during the trip. They were asked 
to write about how they were preparing for the trip, what their im- 
pressions of the United States were before they left and after they had 
been there, and what they did in the United States. Though they were 
assigned to write one page per week during the orientation and two 
pages per week during the trip, most students wrote much more. Di- 
rect quotations in this paper are from students’ journal entries. 

Students’ Preparation Outside of Class 

According to their responses to the questionnaire described below, 
most students did additional preparation for their overseas trip. The 
types of preparation they listed included: 1) reading books or maga- 
zine articles (about the United States, homestay, and so on), 2) listen- 
ing to radio or television English conversation programs, 3) watching 
foreign television programs or movies, 4) listening to English tapes, 
5) attending conversation classes, and 6) watching television news in 
English. Only one student responded that she did not do any prepara- 
tion outside of the orientation class. 

The Mary Baldwin College Program 

Classes 

The students are at Mary Baldwin College three weeks, during 
which time they take American culture classes (in 1992, American 
children’s culture, religions in America, and American history) and 
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English language classes (reading, writing and speaking/listening). 
The culture classes emphasize student participation and discussion 
rather than lecture. Among other assignments, students did a short 
independent study project for the culture classes, made journal en- 
tries in class for the composition class, interviewed local business- 
people and college employees for the listening/speaking class, and 
wrote a report on that experience for their composition class. Fables, 
myths, and poems written by students for the composition class were 
gathered in a book that the students were given at the end of the course. 

Students found the classes interesting and enjoyable. One student 
wrote, “We have our first class yesterday. I felt that an hour was very 
short, because every class was interesting for me.” Another student 
wrote, “In Japan, most classes which I have in college are passive 
classes except for English classes. I only write down what a teacher 
writes on blackboard, but in America, I can speak in class, so I can 
feel that I am taking part in class.” 

Living Situation 

During the time they were at MBC, students lived in a dormitory. 
They were divided into four groups, and there was a counselor, an 
American college student or recent graduate, assigned to each group. 
Students ate meals with their counselors and spent time with them 
outside of class. Though there were other Japanese students on cam- 
pus at the same time, the Doshisha students had little contact with 
them. 

Field Trips 

In addition to classes, there were a number of field trips. Students 
visited a local organ factory, Richmond (including an art museum and 
a history museum), Williamsburg, Jamestown, Woodrow Wilson’s 
Birthplace, a local dairy farm and poultry farm, Monticello, and the 
Museum of American Frontier Culture (where they could see farms 
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from the 18th-century United States and three European countries), 
and attended Jazz in the Park and the Oak Grove Music Festival. 

Homestay 

After three weeks at MBC, students had a homestay experience 
with local families. Students were assigned to homestay families in 
pairs. They first met their families at a dinner and then spent four and 
a half days with them. With the homestay families, students engaged 
in a wide variety of activities, including camping, hiking, swimming, 
horseback riding, visiting museums, attending church services and 
other church activities, and visiting the host families’ relatives or neigh- 
bors. 

Entries in the students’ journals indicated that the homestay expe- 
rience was a universally positive one, and that all of the students formed 
good relationships with their host families and felt that, as a result of 
the experience, they learned a great deal about American daily life, 
particularly family life. One student wrote of the experience, “Though 
I stayed with them for only five days, it was the most wonderful time 
for me. They were very sweet and treated us as treasures. I could 
touch American daily life directly. I discovered many things.” An- 
other wrote, “They were all very kind and treated us as members of 
their family. I was very glad to stay with them a week. I won’t forget 
them.” 



III. Short-Term Overseas Study Program 171 



Appendix B 

Overseas Orientation Survey (May 1992) 

Answer the following questions. If you have difficulty answering 
in English, please answer in Japanese. 

Name: Age: Years ^Months 

1. Have you ever been to the United States? If so, how long were 
you there? What did you do while you were there? 

2. Have you ever been to any other foreign country? If so, how 
long were you there? What did you do while you were there? 

3. Rate your English proficiency on a scale of 0-10. (10 means as 
proficient as a native English speaker; 0 means no ability at all.) Circle 
the number. 

01234 5 6789 10 

4. What is your image of the United States, both negative and posi- 
tive? Write five positive (good) adjectives and five negative (bad) 
adjectives. Briefly explain with what you mean by each. 

Positive: 

1 . 

2 . 

3. 

4. 

5 . 

Negative: 

1 . 

2 . 

3. 





6 
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4. 

5. 



Overseas Orientation Survey (October 1992) 

Answer the following questions. If you have difficulty answering 
in English, please answer in Japanese. 



1 . In addition to attending the overseas orientation class, how did 
you prepare for the trip to the United States? (for example, radio or 
TV English conversation programs; conversation classes; reading 
books)? 

2. Rate your English proficiency after the trip on a scale of 0-10. 
(10 means as proficient as a native English speaker; 0 means no abil- 
ity at all.) Circle the number. 

0 1 2 3 4 5 6 7 8 9 10 

3 . What is your image of the United States, both negative and posi- 
tive? Write five positive (good) adjectives and five negative (bad) 
adjectives. Briefly explain with what you mean by each. 

Positive: 

1 . 

2 . 

3. 

4. 

5. 

Negative: 

1 . 

2 . 

3. 



Name: 




4. 

5. 
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4. How did your image of the United States change as a result of 
your time there? 

5. What was your image of American families before your 
homestay? How did your image of American families change as a 
result of your homestay? 

6. In this class you prepared for the trip to the United States in 
various ways. 

1) learning about functions (requests, greetings, apologies, etc.) — 
Communicating with Americans 

2) learning about American culture — Background to the USA 

3) learning about traveling and using English — Communicating in the 
US. 

4) presentations — about traveling, American culture and the places 
that we would visit 

5) learning about English expressions — Kanko kara business made: 
Travel eikaiwa 

6) Other (please specify) 

Which did you find most useful (write in the number)? 

Which did you find least useful (write in the number)? 

7. What advice would you give to next year’s director for the MBC 
program? What kinds of preparation did you find especially useful? 
What kinds did not help? What did you wish you had known? 

8. Which city did you enjoy visiting most? Which city did you not 
enjoy visiting? 





Appendix IV 

Doing a Presentation on Your Research 

Aside from writing your research as a research paper or report, 
you might have to report on it orally. You might do a report of your 
research for a committee of your co-workers. You might do a presen- 
tation at a conference. You might explain the results of your research 
to your supervisor. We will discuss some of the issues you need to 
think about before you prepare for the presentation, how to prepare 
for your presentation, and as you give your presentation. 

Preparation 

Before You Prepare for Your Presentation 

Before you even start to prepare for your presentation, there are 
some questions that you need to consider. You need to keep these in 
mind as you prepare for the presentation. 

The purpose of your presentation. Before you even start to pre- 
pare your presentation, you need to think about its purpose. When 
you are planning a presentation, the first thing you need to do is to 
make the purpose of the presentation clear in your own mind. This is 
very obvious, but it is sometimes neglected. It is useful to write down 
in a single sentence what you plan to accomplish with your presenta- 
tion. You may or may not actually use this sentence in your presenta- 
tion, but you should keep it in front of you as you work on your pre- 
sentation. 

In presenting a research paper, the purpose is usually to report the 
results of the research. Which of the results should you emphasize? 
How can you present them most clearly? How much theoretical back- 
ground is necessary to understand the results? (This depends, in part 
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on the audience, discussed below.) 

Considering the audience. You need to consider the audience for 
the presentation, As much as possible, you need to know your audi- 
ence. How old are they? Are they men or women? What do they al- 
ready know about the topic? What are their beliefs? Their social and 
economic backgrounds? Why have they come to listen to you? You 
should have as much information as possible about your audience, 
and you should keep it in mind as you prepare your presentation. This 
is called audience analysis. 

If your audience does not know much about the subject, you will 
have to explain the background. If they are very familiar with the 
subject, you will not have to explain it. 

You also need to consider how the audience will use the informa- 
tion. If you are doing a presentation on an aspect of language to a 
group of language teachers, you will probably want to emphasize as- 
pects of your research that will be useful in their classrooms. If you 
are giving the presentation to a group of linguists, they will probably 
be most interested in how your results support or contradict a theory 
of linguistics. 

In some cases, you may speak to an audience whose composition 
you do not know. If the setting is not too formal, you might want to 
start by asking them how familiar they are with the topic, what their 
backgrounds are, etc. This will allow you to adapt your presentation 
to some degree. 

Preparing Your Presentation 

Written and spoken communication. As you prepare your presen- 
tation, you should keep in mind that written and spoken communica- 
tion have differences. One of the most important is that a reader can 
go back and re-read a sentence or a paragraph several times if the 
meaning is not clear. However, a listener does not have that luxury. 
Therefore, it is not a good idea just to read the written version of your 
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paper out loud. You should use shorter, simpler sentences when you 
speak. You should choose a few points that you want to make and 
concentrate on those points. You should make certain that everything 
included in your talk contributes to making those points. Also, do not 
be afraid to repeat your important points more than once. It is easier 
for listeners to miss them than for readers to miss them. 

Organizing your presentation. Because of this difference, it is par- 
ticularly important that your presentation be well organized. A gen- 
eral principle of public speaking is that first you tell the audience 
what you are going to tell them (introduction); then you tell them 
(discussion or body); then you tell them what you told them (conclu- 
sion). That is, first, you let the audience know what the presentation 
will be about — what the main sections of the presentation will be 
and what topic areas you will cover. Second, you present the back- 
ground and research results. Finally, you explain your conclusions 
and summarize the main ideas. You can use this organization for the 
presentation as a whole and also for each major section of the presen- 
tation. 

Generally you will write the body of the presentation first. After 
you write the body of your presentation, you write the conclusion and 
then the introduction. The conclusion should summarize the main 
points and emphasize what you hope to accomplish with your presen- 
tation. Your audience is likely to remember the points that you make 
last. Finally, you need to write your introduction. In the introduction, 
you need to capture the attention of your audience and make the topic 
and scope of your presentation clear. 

It will also make your presentation clearer if you include examples 
or quotations. Consider your audience and what kinds of examples 
and quotation they are likely to find easy to understand and which 
will help make your points clearer to them. 

Do not try to present all of your research results. Choose the most 
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interesting results, and the ones that are most relevant to your audi- 
ence. Also, do not try to show the audience detailed charts with many 
numbers. You can put emphasis on what your results mean (the con- 
clusion of your research paper) rather than the numbers themselves. 

Using Visuals. Even though you are giving an oral presentation, 
you do not have to depend entirely on oral communication. You can 
use an overhead projector, a handout, or a computer program called 
Power Point. You can include an outline of your presentation, phrases 
expressing your main ideas, charts or graphs presenting your statis- 
tics, etc. However, you should design this visual part of your presen- 
tation carefully, so that each page or slide is not too crowded, and it is 
easy for the audience to pick out the important points. The print should 
be large enough that it is easy to read, even in the back of the room. In 
some cases, you might also use videos or other types of audio or vi- 
sual aids. 

You should make certain that the visuals that you use help support 
the main points you want to make. If you overuse them, or if you use 
them only for the sake of using them, they will be ineffective and 
distracting. Your time is usually limited, and using audio-visual aids 
takes time. You will have to be well prepared in advance and make it 
sure that any equipment you are using works properly before your 
presentation. 

Using appropriate language. As you write your presentation, you 
need to choose appropriate language. Based on the occasion at which 
you are speaking and on your analysis of the audience, you need to 
decide how formal the language you use should be, whether you can 
include technical terms, what sorts of illustrations and data you can 
use, etc. If the presentation is at a meeting of professional people, or if 
it is done for your co-workers, you can and should use technical terms, 
because they will help you get your message across accurately and 
clearly. However, if your audience is made up of non-professionals 
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who are not likely to be familiar with technical terms, you should 
avoid them. At a presentation that is informal, for example, one for a 
group of colleagues or at a small conference, you can speak casually 
and include jokes. 

Involving the audience. In some cases, you may be speaking in an 
informal situation where you can involve the audience. You can plan 
questions to ask the audience to help them see how your topic applies 
to them. This is especially useful if you are speaking to people who 
are not specialists. 

Practicing. When you have finished writing your presentation, you 
need to practice in order to be well prepared — though you should not 
practice so much that you sound stiff. First you can practice by your- 
self. Speak at the speed you will use for your presentation and time 
yourself. (Keep in mind that you may get behind and need to cut parts 
of your presentation; think about what might be able to be cut.) If you 
have a large audience, your rate of speaking needs to be slower. You 
need to make certain that you can finish your presentation within the 
time allowed, but that you do not have too much time left over. (De- 
pending on the setting where you are speaking, you may need to al- 
low time for questions or discussion at the end.) Ask a friend to listen 
to your presentation and comment on it. Practice gives you confi- 
dence, helps control stage fright, and increases your source credibil- 
ity by allowing you to sound confident. 

Doing Your Presentation 
Establishing Your Credibility 

As you deliver your presentation, you need to establish yourself as 
a credible source. Source credibility is related to whether the source 
seems believable and trustworthy. Source credibility can be demon- 
strated in a wide variety of ways. Showing that you are familiar with 
the topic, showing that you are a trustworthy and concerned person. 
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wearing appropriate clothes, referring to your qualifications to speak 
about the topic, and speaking in a self-confident and self-assured 
manner can add to your source credibility. 

Audience Feedback 

Pay attention to the response of the audience. Watch their faces as 
you speak. Do they seem to understand? If not, you may need to slow 
down, explain a point in a different way, etc. Are they getting rest- 
less? Maybe you should go on to the next point. 

Although the audience cannot “re-read,” they may have the oppor- 
tunity to give feedback. Unless you are doing a presentation in a very 
formal situation, or to a very large audience, invite your audience 
members to ask questions. As you begin your presentation, you might 
invite the audience members to ask questions whenever they have 
any. During natural breaks in the presentation — for example, at the 
end of a section — you can ask if anyone has any questions. 

Delivery 

The way you deliver your presentation is almost as important as 
the content. As you deliver the presentation, you need to speak slowly 
but fluently and clearly enough that your audience can understand 
you. Be aware that if you are nervous when you are delivering your 
presentation, you are likely to speak faster than normal. As you are 
giving your presentation, you may need to make a conscious effort to 
slow down. You need to speak loudly enough that the entire audience 
can hear you, but not too loudly. You can use changes in the loudness 
of your voice to emphasize your important points. You should vary 
the tone of your voice to help keep your audience’s attention. You 
should stand in a relaxed posture, except in the most formal situa- 
tions, and your gestures should be natural and support the content of 
your presentation, for example, by using hand gestures to emphasize 
important points. You should maintain eye contact with members of 
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your audience as you speak, rather than looking at the wall in the back 
of the room or the floor in front of you. 

You should also watch the time as you speak. If you seem to be 
getting behind, you may need to cut parts of the presentation. If you 
are going more quickly than you expected, you may need to slow 
down, give additional examples, ask if anyone has any questions, etc. 

Checklist 

As you are preparing for and doing the presentation, keep in mind 
the following: 

I. Preparation 

A. Before you begin 

1 . Consider the purpose 

2. Consider the audience 

B. Doing the preparation 

1 . Keep in mind the differences between written and spoken com- 
munication 

a. Keep sentences short and simple 

b. Concentrate on a few points 

c. Repeat the most important points 

2. Organize your presentation 

a. Include an introduction, discussion and conclusion 

b. Write the body first, then the conclusion, and finally the 
introduction 

3. Consider how to make the presentation clearer by using visuals 

a. Visuals should be easy to read and uncrowded 

b. Choose your visuals carefully and do not overuse them 

4. Consider the language you are going to use 

a. Decide how formal to be 

b. Decide how technical to be 
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c. Plan ways to involve your audience, if appropriate 
5. Practice 

a. Consider the pace of your speaking 

b. Consider the amount of time you have 

c. Ask a friend to listen and give you comments 
III. Doing the presentation 

1 . Establish source credibility 

2. Pay attention to the audience response 

3. Invite the audience to ask questions, if appropriate 

4. Consider your delivery 

a. Be aware of your pace; consciously slow down, if necessary 

b. Vary the tone of your voice and loudness 

c. Stand in a relaxed posture 

d. Use natural gestures 

e. Maintain eye contact 

5. Be aware of the time 

Conclusion 

Many people are nervous about giving presentations, especially if 
they do not have much experience. However, if you think carefully 
about your audience, consider what you want to communicate to them, 
organize your presentation well, practice, and pay attention to the 
audience’s feedback, you can give a good presentation. 

Application Questions 

Imagine that you are presenting your research results in a 20-minute 
presentation at a conference. What kind of audience would you have? 
What aspects of your presentation should you emphasize? What kinds 
of visuals will you use? 



Appendix V 

Titles, Abstracts, and Biodata 

When you write a paper, you also must give it a title and often 
write an abstract (a brief summary) of the paper and biodata (infor- 
mation about yourself). The title and abstract help others find out what 
the paper is about and whether they are interested in reading it, or, in 
the case of an abstract that is part of a conference proposal, it helps 
the program committee decide whether to accept the presentation. 
Therefore, it is important to write them effectively. The biodata helps 
establish your credibility as a source and lets others know what your 
background and qualifications are. 

Titles 

The title obviously gives information about what your paper will 
be about. In a very few words, it needs to let the reader know what the 
study is about. The title should: 

1 . give the topic of the study 

2. indicate the scope of the study (may include the type of partici- 
pants in the study [for example, Japanese, American university stu- 
dents], the type of setting where you did the study [a medium-sized 
company, an elementary school], etc.) 

3. the type of study it is (experiment, case study, preliminary study, 
etc.) (optional) 

Titles should not give the results of the study, just tell what the 
study was about. Some examples of titles are: 

Reducing University Students’ Test Anxiety 

Japanese Students’ Errors in the Use of the Present Tense 

The Use of Sentence Connectors in Academic English 
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“Sorry, I’m Busy”: Refusals of Invitations in British English and 
Japanese English 

Horizontal Communication in a Medium-Sized Company: A Case 
Study 



Abstracts 

When you write a research paper or a proposal for a presentation 
at a conference, you may need to write an abstract of your paper. An 
abstract is a short summary (usually between 100 and 300 words) of 
the paper you have written or plan to present. Your abstract is very 
important. If it is for a published paper, it may be the only part of the 
paper many people read. If the abstract interests them, they may read 
the entire article. If it is part of a proposal for a presentation, it will be 
used to decide whether your presentation is accepted. 

Whenever you write an abstract, whatever its purpose, you should 
check to find out how long it should be. The instructions for the pro- 
posal or paper should tell you how long the abstract should be, and 
you should be careful to stay within the stated limit. (Use the word 
counter under “tools” in Microsoft Word to check the number of 
words.) On the other hand, the abstract should not be significantly 
shorter than the stated limit. For example, if the limit is 200 words, a 
100-word abstract is too short. It should be at least 175 words. 

Research Paper Abstracts 

Depending on the format of the journal where your paper is pub- 
lished, or your teacher or supervisor’s assignment, you will often need 
to write an abstract for a research paper Research paper abstracts are 
generally short, rarely longer than 200 words and usually four to ten 
sentences. 

An abstract for a research paper should have one to three sentences 
for each of the sections of the paper (literature review, methods, re- 
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suits, discussion/conclusion). The abstract should include briefly: 

1 . information about the context of the research and what the pur- 
pose of the research was (that is, what the researcher was trying to 
learn) 

2. how the researcher did the research (whether information was 
gathered by questionnaire, interview, or observation; who the partici- 
pants were; etc.) 

3. what the most important results were 

4. what the results mean 

Abstracts usually use the impersonal passive. For example, in an 
abstract, you would use “A questionnaire was administered” rather 
than “We/The researchers administered a questionnaire.” However, 
some writers use expressions like “We discussed...,” “I showed...,” 
“We argued...,” and “I concluded....” 

Conference Proposal Abstracts 

If you want to do a presentation at a conference, you will probably 
need to write an abstract as part of the proposal. The abstract is read 
by the program committee of the conference in order to decide whether 
to accept the proposal. 

A conference abstract is in generally the same format as a research 
paper abstract. However, there are a few differences. A conference 
proposal abstract may be longer than one for a research paper, so you 
can make some parts of the abstract longer. However, you do not nec- 
essarily have to report your specific results or conclusions as part of 
the abstract for a conference proposal. (Conference proposals are of- 
ten submitted six months or more before the conference, and your 
research might not be complete.) 

The emphasis of a conference proposal should be in three areas. 
You want to: 

1. show the program committee that you know the subject area 
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that you are proposing to give a presentation on 

2. make it clear what your presentation will be about, particularly 
what hypotheses or research questions you are working on 

3. show that your presentation will be interesting to the audience at 
the conference you are submitting the proposal for (Before you even 
write the abstract, of course, it is important to consider what type of 
conference you are submitting your proposal for and how your pa- 
per is appropriate for the people who will attend that conference.) 

One way to show in a conference proposal abstract that you are 
familiar with the research or theory in your subject area is to include 
a carefully chosen reference to one or two important papers in the 
subject area. For example, you might write, “As Smith and Jones 
(2000) have shown....” Then at the end of your abstract, you include 
the citation for the paper by Smith and Jones, using the style of cita- 
tions discussed in Appendix II. However, this should not be a major 
part of your abstract. 

Biodata 

If your paper is accepted for presentation at a conference or publi- 
cation in a journal, you may be asked to provide information about 
yourself. The number of words will be specified (generally less than 
30 words), and some journals and conferences ask for specific infor- 
mation, or ask you not to include certain information. Generally, you 
include your full name, your relevant educational background, your 
employer and position, books published, if any, and your interests. 
Biodata is written in the third person. 

For example: 

Alan Smith received his Ph.D. in communication from Michigan State 
University in 1 989. He is a professor at the University of Kansas and 
does research in and teaches interpersonal communication. 
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Conclusion 

While the title, abstract, and biodata may be among the last parts 
of paper that you write, they are still very important. The title and 
abstract let readers know what your paper is about and helps them 
know whether they would like to learn more. The biodata provides 
information about your background and lets the reader know whether 
you are a believable source. 

Application Exercise 

Reduce your literature review, methods section, results, and dis- 
cussion/conclusion each to one or two sentences. Put them together 
and write connectors so the sentences read smoothly to make an ab- 
stract for your paper. 

What is the topic of your study? What is its scope? What type of 
study is it? Using this information, what title are you going to give 
your paper? 

Discussion Questions 

Look at the abstract of the paper in Appendix III. What informa- 
tion does it have about the background and literature review, the meth- 
ods, the results, and the conclusion? 
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Appendix VI 

Answers for Exercises 

Application Questions 
Chapter 14 

t-shirt group suit group 
range 2 3 

mean 4.0 2.9 

s.d. 0.82 0.99 

The range is greater for the suit group; the mean is higher for the t- 
shirt group, which means that they rated the woman as more friendly 
than the suit group did. The standard deviation is larger for the suit 
group, which means that their responses were more spread out. 

Chapter 15 
TTest 

The t value is 2.70, and the P = .007. Therefore, the difference is 
significant. The woman is judged as significantly more friendly when 
she is wearing jeans and a t-shirt than when she is wearing a suit. 

Chi Square 

Cases 



Category Observed Expected Residual 



A 


4 




10.00 


-6.00 


B 


19 




10.00 


9.00 


C 


7 




10.00 


-3.00 


Total 


30 








Chi-Square 




D.F. 




Significance 


12.6000 




2 




.0018 
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Advertisement B was judged as being significantly more effective 
than the others. 

Pearson Correlation 

The correlation is .94, and p > .0001 , so there is a significant correla- 
tion between vocabulary knowledge and reading proficiency. 



ANOVA 



Source of Variation 


Sum of 
Squares 


DF 


Mean 

Square 


F 


Sig 
of F 


Main Effects 


2779.400 


2 


1389.700 


10.183 


.001 


NATIONALITY 


1960.200 


1 


1960.200 


14.363 


.002 


TEST 


819.200 


1 


819.200 


6.003 


.026 


2-Way Interactions 


12.800 


1 


12.800 


.094 


.763 


NATIONAL TEST 


12.800 


1 


12.800 


.094 


.763 


Explained 


2792.200 


3 


930.733 


6.820 


.004 


Residual 


2183.600 


16 


136.475 






Total 


4975.800 


19 


261 .884 







There are significant effects for both the nationality and the test (lis- 
tening and speaking ability). There is no interaction effect. The Ger- 
man speakers did significantly better on the tests, and reading scores 
were significantly higher than the listening scores. 

Discussion Questions 
Chapter 4 

The paper has elements of both quantitative and qualitative research. 
For example, the writer uses statistics to compare the students’ rat- 
ings of their English proficiency before and after the study. The anal- 
ysis of the students’ journals made use of qualitative methods, dis- 
cussing the students’ experience of the program. 
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Chapter 5 

The researcher considers the variables of the students’ English pro- 
ficiency and their impressions of the US. 

The research questions are: 

1. What kinds of preparation are useful for students planning to go 
overseas? 

2. How did a short-term overseas program in the US affect Japanese 
students’ English language proficiency? 

3. How did a short-term overseas program in the US affect Japanese 
students’ impressions of the US? 

Chapter 6 

She operationalizes the construct of English proficiency as the stu- 
dent’s rating of her English proficiency and the counselors’ observa- 
tions about the students’ English proficiency. 

She operationalizes the students’ perception of the US as the posi- 
tive and negative adjectives listed by the students. 

Chapter 7 

She uses a questionnaire, students’ journals, and observations by 
the counselors. The questions about the students’ perceptions of the 
US allow the students to write about what is uppermost in their mind, 
rather than just responding to the researcher’s adjectives (for exam- 
ple, in a semantic differential). The students’ journals similarly allow 
students to express their experiences in their own way, but they are 
very time consuming to analyze. The counselors have opportunities 
to observe the students using their English in a variety of circum- 
stances. However, the students may react differently to different coun- 
selors, and the counselors are not observing in any organized way. 

Chapter 8 

The questionnaire used short-answer questions, a semantic differ- 
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ential, and essay questions. Using a semantic differential instead of 
multiple choice for the most useful preparation might have yielded 
more useful information. 

Chapter 9 

The researcher could have interviewed the students about their im- 
pressions of American culture. This would have provided more infor- 
mation about their impressions before and after the trip and how 
those impressions had changed as a result of their time in the US. 

Chapter 10 

The researcher used the informal observations of the counselors to 
find out how the students’ English proficiency improved. 

Chapter 11 

The study has good external validity for short-term trips overseas, 
because it is using students who are actually experiencing an over- 
seas trip. However, it might not generalize well to a long-term expe- 
rience overseas. Also, all of the students were female, so it might not 
generalize well to males. As for internal validity, the students were 
chosen for the course, but they might be considered an intact group, 
and they might have influenced each other. Since the students vol- 
unteered for the trip, they were self-selected, and they might react 
differently than students who were chosen at random. 

Chapter 12 

The study has good external validity for short-term trips overseas, 
because it is using students who are actually experiencing an over- 
seas trip. However, it might not generalize well to a long-term expe- 
rience overseas. Also, all of the students were female, so it might not 
generalize well to males. As for internal validity, the students were 
chosen for the course, but they might be considered an intact group, 
and they might have influenced each other. Since the students vol- 
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unteered for the trip, they were self-selected, and they might react 
differently than students who were chosen at random. 

Chapter 13 

The researcher does not use a control group, and therefore, there is 
no comparison for changes in attitudes or improvement of English 
proficiency. 

Chapter 15 

The researcher uses a t test to compare the students’ ratings of their 
English proficiency before and after the program. The results indicat- 
ed that the students felt that their English proficiency had improved 
during the program. 

Chapter 16 

The researcher uses the students’ journals to try to understand how the 
students understood and thought about their experiences in the US. 

Chapter 18 

The researcher could have observed the students’ interactions with 
their counselors, teachers, and with other people in the community 
during their stay at Mary Baldwin College to see whether and how 
their English proficiency and confidence improved. 

Appendix I 

In the introduction, the writer gives a little background about the 
overseas program and explains what will be included in the paper. 

In the literature review, the writer describes the results of a few other 
studies of overseas programs. She also justifies the importance of 
orientation programs before going overseas. Finally, she lists three 
research questions. 

In the methods section, the writer gives an overview; describes 
the participants and how they were chosen; describes the measure- 
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merit instrument (which appears in full in the appendix), and describes 
the procedures and the analyses. 

In the results section, the writer reports on the results, based on 
the questionnaire, and interviews with the counselors. 

In the discussion and conclusion, the writer discusses these re- 
sults and added information about the students’ thoughts and expe- 
riences from their journals. 

The appendices include a description of the program and materi- 
als used and reproduce the questionnaires. 

Appendix II 

1 . The writer has a direct quotation on p. 154. She gives the au- 
thor, year, and page number, which refer to the list of references. 

2. The writer quotes Yashima and Viswat in saying that there has 
been little study of overseas experiences. She also quotes the re- 
sults of three studies. She quotes Grove to justify the importance of 
orientation courses. 

3. The author quotes both book and articles. There are also refer- 
ences for the textbooks used in the orientation course. 

Appendix V 

The writer describes the overseas program briefly, explains what the 
literature review covered, describes the methods of data gathering, 
gives two of the results, and mentions what is in the discussion and 
the appendices. 
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